9:50 p.m.
This is a multi-part message in MIME format.
--------------020907010503000006020103
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
Bill,
I have done exactly what you're looking to do. I was trying to mimic
vSAN as well. They had VSA for a while which was acceptable licensing
costs, but that was replaced with vSAN which is ridiculously expensive
for an extra-small business.
I have a four node cluster with 1TB of storage each. Gluster is
configured with replica 4. So basically I have 1TB of usable storage,
which is fine for my needs. VM migration, Gluster replication, hosted
engine and all that works fine. Performance is generally fine, even with
only dual LACP bonded 1GbE NICs in each node. I can do what I want with
networking to fit our NOC and office network environment.
I have been playing with this for about three weeks. Over the weekend, I
had a handful of VMs running, including the hosted engine. They were
pretty much sitting idle doing nothing. I came in Monday and found
everything offline. No power outages, network/switch didn't fail or
reboot, none of the hardware reset. What I found happened was that
glusterd went nuts on two of the nodes (I have no idea why). Gluster was
spitting out logs like crazy, /var filled up, then RAM and swap was
depleted. Two gluster processes/hosts offline out of the four meant
quorum broke, everything came to a halt. I was unable to recover the
gluster logs since they had to be deleted to free up space in /var.
I was able to get everything fixed and back online in about 2-1/2 hours.
So this is impossible to put into production. Storage is the weakest
link and mostly likely to fail, which in this case it did.
I do not recommend this configuration at all. A dedicated machine for
the engine, dedicated hypervisors and dedicated storage nodes or a SAN
is needed for anything beyond experimentation.
I have zero knowledge of VMWare's vSAN, other than basic concepts, so I
cannot say the results would be similar.
On 2/18/2015 5:32 AM, Bill Dossett wrote:
Hi,
Im in at the deep end
Looking for some advice on if this is possible and what version I
should try I with if so.
We are a heavily VMware oriented company
but I am trying to get
RHEV/oVirt in the door. Honestly I would prefer oVirt, but if they
insist on enterprise support I go to Redhat.
So I have had a play with VMware VSAN and know how it works. I am
trying to more or less replicate this setup using open source.
So VSAN uses 3 or more ESXi nodes, with local harddisks as the storage
medium as a virtual SAN. The SAN performance is quite high as you put
at least 1 SSD in each node.
What is nice is you dont have the NAS element here. VMs are on the
local storage, which is partially SSD, so performance is quite good.
I went to a Redhat presentation on Redhat storage and GlusterFS and
basically this acts as a big software defined NAS which does some
pretty cool things, but thats not exactly what I need.
I would like to build oVirt on top of CentOS Oss
that have local
storage in them that is distributed and redundant in the event of a
node failure.
And I probably need to try and build this in a lab under Fusion on my
Mac Book pro to begin with anyway (that bit Im farily confident with)
and if I get that working can probably get some older kit to try it
out for real as a PoC to few people.
So, Ive set up oVirt before that should be ok, I havent setup
Gluster is there any documents that would help me down this road and
make sure I start out using the best version.
Any advice or pointers would be gratefully received.
Thanks
*Bill Dossett*
Systems Architect
*Tech Central Global Engineering Services*
*T* +44 (0)1923 279353
*M* +44 (0)777 590 8612
b <mailto:jeff.gonzalez@pb.com>ill.dossett@pb.com
<mailto:ill.dossett@pb.com>
pitneybowes.com
*Pitney Bowes*
6 Hercules Way | Leavesden | WD25 7GS | UK
Learn more about Global Engineering Services
<http://teamsite.inside.pb.com/sites/TCPPM/GES/SitePages/Home.aspx>
In Engineering?
*Raise a ticket via Remedy Anywhere [HERE
<teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident/newInciden...
less than a minute*
**
*CloudForms User Guide available* [*HERE
<https://dl.dropboxusercontent.com/u/6219441/ManageIQ%20-%20User%20Manual%...]
------------------------------------------------------------------------
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--------------020907010503000006020103
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Bill,<br>
<br>
I have done exactly what you're looking to do. I was trying to mimic
vSAN as well. They had VSA for a while which was acceptable
licensing costs, but that was replaced with vSAN which is
ridiculously expensive for an extra-small business.<br>
<br>
I have a four node cluster with 1TB of storage each. Gluster is
configured with replica 4. So basically I have 1TB of usable
storage, which is fine for my needs. VM migration, Gluster
replication, hosted engine and all that works fine. Performance is
generally fine, even with only dual LACP bonded 1GbE NICs in each
node. I can do what I want with networking to fit our NOC and office
network environment.<br>
<br>
I have been playing with this for about three weeks. Over the
weekend, I had a handful of VMs running, including the hosted
engine. They were pretty much sitting idle doing nothing. I came in
Monday and found everything offline. No power outages,
network/switch didn't fail or reboot, none of the hardware reset.
What I found happened was that glusterd went nuts on two of the
nodes (I have no idea why). Gluster was spitting out logs like
crazy, /var filled up, then RAM and swap was depleted. Two gluster
processes/hosts offline out of the four meant quorum broke,
everything came to a halt. I was unable to recover the gluster logs
since they had to be deleted to free up space in /var.<br>
<br>
I was able to get everything fixed and back online in about 2-1/2
hours. So this is impossible to put into production. Storage is the
weakest link and mostly likely to fail, which in this case it did.<br>
<br>
I do not recommend this configuration at all. A dedicated machine
for the engine, dedicated hypervisors and dedicated storage nodes or
a SAN is needed for anything beyond experimentation.<br>
<br>
I have zero knowledge of VMWare's vSAN, other than basic concepts,
so I cannot say the results would be similar.<br>
<br>
<br>
<div class="moz-cite-prefix">On 2/18/2015 5:32 AM, Bill Dossett
wrote:<br>
</div>
<blockquote
cite="mid:B47F00825D68444ABC037330782753603771A364@013-BR1MPN1-012.MGDPBI.global.pvt"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Im in at the deep end
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Looking for some advice on if this is
possible and what version I should try I with if
so.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are a heavily VMware oriented company
but I am trying to get RHEV/oVirt in the door. Honestly I
would prefer oVirt, but if they insist on enterprise support I
go to Redhat.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So I have had a play with VMware VSAN and
know how it works. I am trying to more or less replicate this
setup using open source.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So VSAN uses 3 or more ESXi nodes, with
local harddisks as the storage medium as a virtual SAN. The
SAN performance is quite high as you put at least 1 SSD in
each node.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">What is nice is you dont have the NAS
element here. VMs are on the local storage, which is
partially SSD, so performance is quite good.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I went to a Redhat presentation on Redhat
storage and GlusterFS and basically this acts as a big
software defined NAS which does some pretty cool things, but
thats not exactly what I need.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I would like to build oVirt on top of
CentOS Oss
that have local storage in them that is
distributed and redundant in the event of a node
failure.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And I probably need to try and build this
in a lab under Fusion on my Mac Book pro to begin with anyway
(that bit Im farily confident with) and if I get that working
can probably get some older kit to try it out for real as a
PoC to few people.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So, Ive set up oVirt before that should be
ok, I havent setup Gluster is there any documents that would
help me down this road and make sure I start out using the
best version.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any advice or pointers would be gratefully
received.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable" style="width:480.0pt"
border="0"
cellpadding="0" cellspacing="0" width="640">
<tbody>
<tr>
<td style="width:480.0pt;padding:0cm 0cm 0cm 0cm"
width="640">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0082D5;mso-fareast-language:EN-GB">Bill
Dossett</span></b><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">Systems
Architect</span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
</td>
</tr>
<tr>
<td style="width:480.0pt;padding:0cm 0cm 0cm 0cm"
width="640">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">Tech
Central Global Engineering
Services</span></b><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;mso-fareast-language:EN-GB"> </span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">T</span></b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB"> +44
(0)1923 279353</span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">M</span></b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB"> +44
(0)777 590 8612</span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-GB"><a
moz-do-not-send="true"
href="mailto:jeff.gonzalez@pb.com"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:purple">b</span></a></span><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0082D5;mso-fareast-language:EN-GB"><a
moz-do-not-send="true"
href="mailto:ill.dossett@pb.com"><span
style="color:blue">ill.dossett@pb.com</span></a></span><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0082D5;mso-fareast-language:EN-GB"><a
moz-do-not-send="true"
href="pitneybowes.com"><span
style="color:#0082D5">pitneybowes.com</span></a></span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;mso-fareast-language:EN-GB"><br>
</span><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">Pitney
Bowes</span></b><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">6
Hercules Way | Leavesden | WD25 7GS | UK</span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB"> </span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
</td>
</tr>
<tr>
<td style="width:480.0pt;padding:0cm 0cm 0cm 0cm"
width="640"><br>
</td>
</tr>
<tr>
<td style="width:480.0pt;padding:0cm 0cm 0cm 0cm"
width="640">
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0082D5;mso-fareast-language:EN-GB"><a
moz-do-not-send="true"
href="http://teamsite.inside.pb.com/sites/TCPPM/GES/SitePages/Home.a...
style="color:#0082D5">Learn more about Global
Engineering Services</span></a></span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"
style="page-break-after:avoid"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB"> </span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"
style="page-break-after:avoid"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">In
Engineering?</span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">Raise
a ticket via Remedy Anywhere [<span
style="color:#0082D5"><a
moz-do-not-send="true"
href="teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident...
style="color:#0082D5">HERE</span></a></span>]<span
style="color:blue"> </span>takes less than a
minute</span></b><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB"> </span></b><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB">CloudForms
User Guide available</span></b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";mso-fareast-language:EN-GB"> [<b><span
style="color:#0082D5"><a
moz-do-not-send="true"
href="https://dl.dropboxusercontent.com/u/6219441/ManageIQ%20-%20Use...
style="color:#0082D5">HERE</span></a></span></b>]</span><span
style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<br>
<hr>
<font color="Black" face="Arial"
size="2"><br>
</font>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
</body>
</html>
--------------020907010503000006020103--