[ovirt-users] ovirt and glusterfs setup

Donny D donny at cloudspin.me
Wed Feb 18 20:21:08 UTC 2015


    
I did not have a good experience putting both gluster and virt on the same node. I was doing hosted engine with replicate across two nodes, and one day it went into split brain hell... i was never able to track down why. However i do have a gluster with distribute and replica setup on its own with a couple nodes, and it has given me zero problems in the last 60 days. It seems to me that gluster and virt need to stay seperate for now. Both are great products and both work as described, just not on the same node at the same time.


Happy Connecting. Sent from my Sprint Samsung Galaxy S® 5

-------- Original message --------
From: George Skorup <george at mwcomm.com> 
Date: 02/18/2015  12:50 PM  (GMT-07:00) 
To: users at ovirt.org 
Subject: Re: [ovirt-users] ovirt and glusterfs setup 


    Bill,

    

    I have done exactly what you're looking to do. I was trying to mimic
    vSAN as well. They had VSA for a while which was acceptable
    licensing costs, but that was replaced with vSAN which is
    ridiculously expensive for an extra-small business.

    

    I have a four node cluster with 1TB of storage each. Gluster is
    configured with replica 4. So basically I have 1TB of usable
    storage, which is fine for my needs. VM migration, Gluster
    replication, hosted engine and all that works fine. Performance is
    generally fine, even with only dual LACP bonded 1GbE NICs in each
    node. I can do what I want with networking to fit our NOC and office
    network environment.

    

    I have been playing with this for about three weeks. Over the
    weekend, I had a handful of VMs running, including the hosted
    engine. They were pretty much sitting idle doing nothing. I came in
    Monday and found everything offline. No power outages,
    network/switch didn't fail or reboot, none of the hardware reset.
    What I found happened was that glusterd went nuts on two of the
    nodes (I have no idea why). Gluster was spitting out logs like
    crazy, /var filled up, then RAM and swap was depleted. Two gluster
    processes/hosts offline out of the four meant quorum broke,
    everything came to a halt. I was unable to recover the gluster logs
    since they had to be deleted to free up space in /var.

    

    I was able to get everything fixed and back online in about 2-1/2
    hours. So this is impossible to put into production. Storage is the
    weakest link and mostly likely to fail, which in this case it did.

    

    I do not recommend this configuration at all. A dedicated machine
    for the engine, dedicated hypervisors and dedicated storage nodes or
    a SAN is needed for anything beyond experimentation.

    

    I have zero knowledge of VMWare's vSAN, other than basic concepts,
    so I cannot say the results would be similar.

    

    

    On 2/18/2015 5:32 AM, Bill Dossett
      wrote:

    
    
      
      
      <!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";
	mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:#0563C1;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:#954F72;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";
	mso-fareast-language:EN-US;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
-->
      
        Hi,
         
        I’m in at the deep end…
         
        Looking for some advice on if this is
          possible – and what version I should try I with if so.
         
        We are a heavily VMware oriented company…
          but I am trying to get RHEV/oVirt in the door.  Honestly I
          would prefer oVirt, but if they insist on enterprise support I
          go to Redhat.
         
        So I have had a play with VMware VSAN and
          know how it works.  I am trying to more or less replicate this
          setup using open source.
         
        So VSAN uses 3 or more ESXi nodes, with
          local harddisks as the storage medium as a virtual SAN.  The
          SAN performance is quite high as you put at least 1 SSD in
          each node.
         
        What is nice is you don’t have the NAS
          element here.  VMs are on the local storage, which is
          partially SSD, so performance is quite good.
         
        I went to a Redhat presentation on Redhat
          storage and GlusterFS and basically this acts as a big
          software defined NAS which does some pretty cool things, but
          that’s not exactly what I need.
         
        I would like to build oVirt on top of
          CentOS Oss… that have local storage in them that is
          distributed and redundant in the event of a node failure.
         
        And I probably need to try and build this
          in a lab under Fusion on my Mac Book pro to begin with anyway
          (that bit I’m farily confident with) and if I get that working
          can probably get some older kit to try it out for real as a
          PoC to few people.
         
        So, I’ve set up oVirt before that should be
          ok, I haven’t setup Gluster is there any documents that would
          help me down this road and make sure I start out using the
          best version.
         
        Any advice or pointers would be gratefully
          received.
         
        Thanks
         
        
          
            
              
                Bill
                      Dossett
                Systems
                    Architect
              
            
            
              
                Tech
                      Central – Global Engineering Services
                 
                T +44
                    (0)1923 279353
                M +44
                    (0)777 590 8612
                bill.dossett at pb.com
                pitneybowes.com
                

                  Pitney
                      Bowes
                6
                    Hercules Way | Leavesden | WD25 7GS | UK
                 
              
            
            
              

              
            
            
              
                Learn more about Global
                        Engineering Services
                 
                In
                    Engineering?
                Raise
                      a ticket via Remedy Anywhere [HERE] takes less than a
                      minute
                 
                CloudForms
                      User Guide available [HERE]
              
            
          
        
         
         
      
      

      
      

      
      

      
      

      _______________________________________________
Users mailing list
Users at ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

    
    

  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150218/1515d808/attachment-0001.html>


More information about the Users mailing list