<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></head><body>
    
<div>I did not have a good experience putting both gluster and virt on the same node. I was doing hosted engine with replicate across two nodes, and one day it went into split brain hell... i was never able to track down why. However i do have a gluster with distribute and replica setup on its own with a couple nodes, and it has given me zero problems in the last 60 days. It seems to me that gluster and virt need to stay seperate for now. Both are great products and both work as described, just not on the same node at the same time.</div><div><br></div><div><br></div><div><br></div><div id="composer_signature"><div style="font-size:85%;color:#575757">Happy Connecting. Sent from my Sprint Samsung Galaxy S® 5</div></div><br><br>-------- Original message --------<br>From: George Skorup &lt;george@mwcomm.com&gt; <br>Date: 02/18/2015  12:50 PM  (GMT-07:00) <br>To: users@ovirt.org <br>Subject: Re: [ovirt-users] ovirt and glusterfs setup <br><br>
    Bill,<br>
    <br>
    I have done exactly what you're looking to do. I was trying to mimic
    vSAN as well. They had VSA for a while which was acceptable
    licensing costs, but that was replaced with vSAN which is
    ridiculously expensive for an extra-small business.<br>
    <br>
    I have a four node cluster with 1TB of storage each. Gluster is
    configured with replica 4. So basically I have 1TB of usable
    storage, which is fine for my needs. VM migration, Gluster
    replication, hosted engine and all that works fine. Performance is
    generally fine, even with only dual LACP bonded 1GbE NICs in each
    node. I can do what I want with networking to fit our NOC and office
    network environment.<br>
    <br>
    I have been playing with this for about three weeks. Over the
    weekend, I had a handful of VMs running, including the hosted
    engine. They were pretty much sitting idle doing nothing. I came in
    Monday and found everything offline. No power outages,
    network/switch didn't fail or reboot, none of the hardware reset.
    What I found happened was that glusterd went nuts on two of the
    nodes (I have no idea why). Gluster was spitting out logs like
    crazy, /var filled up, then RAM and swap was depleted. Two gluster
    processes/hosts offline out of the four meant quorum broke,
    everything came to a halt. I was unable to recover the gluster logs
    since they had to be deleted to free up space in /var.<br>
    <br>
    I was able to get everything fixed and back online in about 2-1/2
    hours. So this is impossible to put into production. Storage is the
    weakest link and mostly likely to fail, which in this case it did.<br>
    <br>
    I do not recommend this configuration at all. A dedicated machine
    for the engine, dedicated hypervisors and dedicated storage nodes or
    a SAN is needed for anything beyond experimentation.<br>
    <br>
    I have zero knowledge of VMWare's vSAN, other than basic concepts,
    so I cannot say the results would be similar.<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 2/18/2015 5:32 AM, Bill Dossett
      wrote:<br>
    </div>
    <blockquote cite="mid:B47F00825D68444ABC037330782753603771A364@013-BR1MPN1-012.MGDPBI.global.pvt" type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      <meta name="Generator" content="Microsoft Word 15 (filtered
        medium)">
      <style><!--
/* Font Definitions */
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        margin-bottom:.0001pt;
        font-size:11.0pt;
        font-family:"Calibri","sans-serif";
        mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:#0563C1;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:#954F72;
        text-decoration:underline;}
span.EmailStyle17
        {mso-style-type:personal-compose;
        font-family:"Calibri","sans-serif";
        color:windowtext;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-family:"Calibri","sans-serif";
        mso-fareast-language:EN-US;}
@page WordSection1
        {size:612.0pt 792.0pt;
        margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
        {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
      <div class="WordSection1">
        <p class="MsoNormal">Hi,<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">I’m in at the deep end…<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">Looking for some advice on if this is
          possible – and what version I should try I with if so.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">We are a heavily VMware oriented company…
          but I am trying to get RHEV/oVirt in the door.&nbsp; Honestly I
          would prefer oVirt, but if they insist on enterprise support I
          go to Redhat.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">So I have had a play with VMware VSAN and
          know how it works.&nbsp; I am trying to more or less replicate this
          setup using open source.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">So VSAN uses 3 or more ESXi nodes, with
          local harddisks as the storage medium as a virtual SAN.&nbsp; The
          SAN performance is quite high as you put at least 1 SSD in
          each node.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">What is nice is you don’t have the NAS
          element here.&nbsp; VMs are on the local storage, which is
          partially SSD, so performance is quite good.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">I went to a Redhat presentation on Redhat
          storage and GlusterFS and basically this acts as a big
          software defined NAS which does some pretty cool things, but
          that’s not exactly what I need.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">I would like to build oVirt on top of
          CentOS Oss… that have local storage in them that is
          distributed and redundant in the event of a node failure.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">And I probably need to try and build this
          in a lab under Fusion on my Mac Book pro to begin with anyway
          (that bit I’m farily confident with) and if I get that working
          can probably get some older kit to try it out for real as a
          PoC to few people.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">So, I’ve set up oVirt before that should be
          ok, I haven’t setup Gluster is there any documents that would
          help me down this road and make sure I start out using the
          best version.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">Any advice or pointers would be gratefully
          received.<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <p class="MsoNormal">Thanks<o:p></o:p></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
        <table class="MsoNormalTable" style="width:480.0pt" border="0" cellpadding="0" cellspacing="0" width="640">
          <tbody>
            <tr>
              <td style="width:480.0pt;padding:0cm 0cm 0cm 0cm" width="640">
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;color:#0082D5;mso-fareast-language:EN-GB">Bill
                      Dossett</span></b><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">Systems
                    Architect</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
              </td>
            </tr>
            <tr>
              <td style="width:480.0pt;padding:0cm 0cm 0cm 0cm" width="640">
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">Tech
                      Central – Global Engineering Services</span></b><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="font-size:10.0pt;mso-fareast-language:EN-GB">&nbsp;</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">T</span></b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">&nbsp;+44
                    (0)1923 279353</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">M</span></b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">&nbsp;+44
                    (0)777 590 8612</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><a moz-do-not-send="true" href="mailto:jeff.gonzalez@pb.com"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;color:purple">b</span></a></span><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;color:#0082D5;mso-fareast-language:EN-GB"><a moz-do-not-send="true" href="mailto:ill.dossett@pb.com"><span style="color:blue">ill.dossett@pb.com</span></a></span><span style="font-size:12.0pt;font-family:&quot;Times New
Roman&quot;,&quot;serif&quot;;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;color:#0082D5;mso-fareast-language:EN-GB"><a moz-do-not-send="true" href="pitneybowes.com"><span style="color:#0082D5">pitneybowes.com</span></a></span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="font-size:10.0pt;mso-fareast-language:EN-GB"><br>
                  </span><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">Pitney
                      Bowes</span></b><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">6
                    Hercules Way | Leavesden | WD25 7GS | UK</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">&nbsp;</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
              </td>
            </tr>
            <tr>
              <td style="width:480.0pt;padding:0cm 0cm 0cm 0cm" width="640"><br>
              </td>
            </tr>
            <tr>
              <td style="width:480.0pt;padding:0cm 0cm 0cm 0cm" width="640">
                <p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;color:#0082D5;mso-fareast-language:EN-GB"><a moz-do-not-send="true" href="http://teamsite.inside.pb.com/sites/TCPPM/GES/SitePages/Home.aspx"><span style="color:#0082D5">Learn more about Global
                        Engineering Services</span></a></span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal" style="page-break-after:avoid"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">&nbsp;</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal" style="page-break-after:avoid"><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">In
                    Engineering?</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">Raise
                      a ticket via Remedy Anywhere [<span style="color:#0082D5"><a moz-do-not-send="true" href="teamsite.inside.pb.com/sites/TCPPM/GES/Lists/Create%20Incident/newIncident.aspx"><span style="color:#0082D5">HERE</span></a></span>]<span style="color:blue">&nbsp;</span>takes less than a
                      minute</span></b><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">&nbsp;</span></b><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
                <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">CloudForms
                      User Guide available</span></b><span style="font-size:10.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;mso-fareast-language:EN-GB">&nbsp;[<b><span style="color:#0082D5"><a moz-do-not-send="true" href="https://dl.dropboxusercontent.com/u/6219441/ManageIQ%20-%20User%20Manual%20PB%20v5.pdf"><span style="color:#0082D5">HERE</span></a></span></b>]</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
              </td>
            </tr>
          </tbody>
        </table>
        <p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Times New
            Roman&quot;,&quot;serif&quot;;mso-fareast-language:EN-GB"><o:p>&nbsp;</o:p></span></p>
        <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
      </div>
      <br>
      <hr>
      <font color="Black" face="Arial" size="2"><br>
      </font>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  </body></html>