[ovirt-users] Users Digest, Vol 71, Issue 32
Moacir Ferreira
moacirferreira at hotmail.com
Mon Aug 7 13:42:14 UTC 2017
Hi Fernando,
Since I have 3 servers then each one will have a connection to the other two. In this case I could setup 3 subnets, one for each link, avoiding in this way to implement bridging (and Spanning Tree) on the servers. My idea is that when I build the server I should use the IP addresses of the 40Gb NICs to setup Gluster and everything should to be ok. Anyway, I will test it on a virtual environment before installing.
Thanks,
Moacir
----------------------------------------------------------------------
Message: 1
Date: Mon, 7 Aug 2017 10:08:32 -0300
From: FERNANDO FREDIANI <fernando.frediani at upx.com>
To: users at ovirt.org
Subject: Re: [ovirt-users] Good practices
Message-ID: <8dd168a0-d5ab-bdf4-5c5d-197909afc515 at upx.com>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"
Moacir, I beleive for using the 3 servers directly connected to each
others without switch you have to have a Bridge on each server for every
2 physical interfaces to allow the traffic passthrough in layer2 (Is it
possible to create this from the oVirt Engine Web Interface?). If your
ovirtmgmt network is separate from other (should really be) that should
be fine to do.
Fernando
On 07/08/2017 07:13, Moacir Ferreira wrote:
>
> Hi, in-line responses.
>
>
> Thanks,
>
> Moacir
>
>
> ------------------------------------------------------------------------
> *From:* Yaniv Kaul <ykaul at redhat.com>
> *Sent:* Monday, August 7, 2017 7:42 AM
> *To:* Moacir Ferreira
> *Cc:* users at ovirt.org
> *Subject:* Re: [ovirt-users] Good practices
>
>
> On Sun, Aug 6, 2017 at 5:49 PM, Moacir Ferreira
> <moacirferreira at hotmail.com <mailto:moacirferreira at hotmail.com>> wrote:
>
> I am willing to assemble a oVirt "pod", made of 3 servers, each
> with 2 CPU sockets of 12 cores, 256GB RAM, 7 HDD 10K, 1 SSD. The
> idea is to use GlusterFS to provide HA for the VMs. The 3 servers
> have a dual 40Gb NIC and a dual 10Gb NIC. So my intention is to
> create a loop like a server triangle using the 40Gb NICs for
> virtualization files (VMs .qcow2) access and to move VMs around
> the pod (east /west traffic) while using the 10Gb interfaces for
> giving services to the outside world (north/south traffic).
>
>
> Very nice gear. How are you planning the network exactly? Without a
> switch, back-to-back? (sounds OK to me, just wanted to ensure this is
> what the 'dual' is used for). However, I'm unsure if you have the
> correct balance between the interface speeds (40g) and the disks (too
> many HDDs?).
>
> Moacir:The idea is to have a very high performance network for the
> distributed file system and to prevent bottlenecks when we move one VM
> from a node to another. Using 40Gb NICs I can just connect the servers
> back-to-back. In this case I don't need the expensive 40Gb switch, I
> get very high speed and no contention between north/south traffic with
> east/west.
>
>
> This said, my first question is: How should I deploy GlusterFS in
> such oVirt scenario? My questions are:
>
>
> 1 - Should I create 3 RAID (i.e.: RAID 5), one on each oVirt node,
> and then create a GlusterFS using them?
>
> I would assume RAID 1 for the operating system (you don't want a
> single point of failure there?) and the rest JBODs. The SSD will be
> used for caching, I reckon? (I personally would add more SSDs instead
> of HDDs, but it does depend on the disk sizes and your space requirements.
>
> Moacir: Yes, I agree that I need a RAID-1 for the OS. Now, generic
> JBOD or a JBOD assembled using RAID-5 "disks" createdby the server's
> disk controller?
>
> 2 - Instead, should I create a JBOD array made of all server's disks?
>
> 3 - What is the best Gluster configuration to provide for HA while
> not consuming too much disk space?
>
>
> Replica 2 + Arbiter sounds good to me.
> Moacir:I agree, and that is what I am using.
>
> 4 - Does a oVirt hypervisor pod like I am planning to build, and
> the virtualization environment, benefits from tiering when using a
> SSD disk? And yes, will Gluster do it by default or I have to
> configure it to do so?
>
>
> Yes, I believe using lvmcache is the best way to go.
>
> Moacir: Are you sure? I say that because the qcow2 files will be
> quite big. So if tiering is "file based" the SSD would have to be
> very, very big unless Gluster tiering do it by "chunks of data".
>
>
> At the bottom line, what is the good practice for using GlusterFS
> in small pods for enterprises?
>
>
> Don't forget jumbo frames. libgfapi (coming hopefully in 4.1.5).
> Sharding (enabled out of the box if you use a hyper-converged setup
> via gdeploy).
> *Moacir:* Yes! This is another reason to have separate networks for
> north/south and east/west. In that way I can use the standard MTU on
> the 10Gb NICs and jumbo frames on the file/move 40Gb NICs.
>
> Y.
>
>
> You opinion/feedback will be really appreciated!
>
> Moacir
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org <mailto:Users at ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170807/f4181b39/attachment-0001.html>
------------------------------
Message: 2
Date: Mon, 7 Aug 2017 15:26:03 +0200
From: Erekle Magradze <erekle.magradze at recogizer.de>
To: FERNANDO FREDIANI <fernando.frediani at upx.com>, users at ovirt.org
Subject: Re: [ovirt-users] Good practices
Message-ID: <aa829d07-fa77-3ed9-2500-e33cc01414b6 at recogizer.de>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Hi Frenando,
Here is my experience, if you consider a particular hard drive as a
brick for gluster volume and it dies, i.e. it becomes not accessible
it's a huge hassle to discard that brick and exchange with another one,
since gluster some tries to access that broken brick and it's causing
(at least it cause for me) a big pain, therefore it's better to have a
RAID as brick, i.e. have RAID 1 (mirroring) for each brick, in this case
if the disk is down you can easily exchange it and rebuild the RAID
without going offline, i.e switching off the volume doing brick
manipulations and switching it back on.
Cheers
Erekle
On 08/07/2017 03:04 PM, FERNANDO FREDIANI wrote:
>
> For any RAID 5 or 6 configuration I normally follow a simple gold rule
> which gave good results so far:
> - up to 4 disks RAID 5
> - 5 or more disks RAID 6
>
> However I didn't really understand well the recommendation to use any
> RAID with GlusterFS. I always thought that GlusteFS likes to work in
> JBOD mode and control the disks (bricks) directlly so you can create
> whatever distribution rule you wish, and if a single disk fails you
> just replace it and which obviously have the data replicated from
> another. The only downside of using in this way is that the
> replication data will be flow accross all servers but that is not much
> a big issue.
>
> Anyone can elaborate about Using RAID + GlusterFS and JBOD + GlusterFS.
>
> Thanks
> Regards
> Fernando
>
>
> On 07/08/2017 03:46, Devin Acosta wrote:
>>
>> Moacir,
>>
>> I have recently installed multiple Red Hat Virtualization hosts for
>> several different companies, and have dealt with the Red Hat Support
>> Team in depth about optimal configuration in regards to setting up
>> GlusterFS most efficiently and I wanted to share with you what I learned.
>>
>> In general Red Hat Virtualization team frowns upon using each DISK of
>> the system as just a JBOD, sure there is some protection by having
>> the data replicated, however, the recommendation is to use RAID 6
>> (preferred) or RAID-5, or at least RAID-1 at the very least.
>>
>> Here is the direct quote from Red Hat when I asked about RAID and Bricks:
>> /
>> /
>> /"A typical Gluster configuration would use RAID underneath the
>> bricks. RAID 6 is most typical as it gives you 2 disk failure
>> protection, but RAID 5 could be used too. Once you have the RAIDed
>> bricks, you'd then apply the desired replication on top of that. The
>> most popular way of doing this would be distributed replicated with
>> 2x replication. In general you'll get better performance with larger
>> bricks. 12 drives is often a sweet spot. Another option would be to
>> create a separate tier using all SSD?s.? /
>>
>> /In order to SSD tiering from my understanding you would need 1 x
>> NVMe drive in each server, or 4 x SSD hot tier (it needs to be
>> distributed, replicated for the hot tier if not using NVME). So with
>> you only having 1 SSD drive in each server, I?d suggest maybe looking
>> into the NVME option. /
>> /
>> /
>> /Since your using only 3-servers, what I?d probably suggest is to do
>> (2 Replicas + Arbiter Node), this setup actually doesn?t require the
>> 3rd server to have big drives at all as it only stores meta-data
>> about the files and not actually a full copy. /
>> /
>> /
>> /Please see the attached document that was given to me by Red Hat to
>> get more information on this. Hope this information helps you./
>> /
>> /
>>
>> --
>>
>> Devin Acosta, RHCA, RHVCA
>> Red Hat Certified Architect
>>
>> On August 6, 2017 at 7:29:29 PM, Moacir Ferreira
>> (moacirferreira at hotmail.com <mailto:moacirferreira at hotmail.com>) wrote:
>>
>>> I am willing to assemble a oVirt "pod", made of 3 servers, each with
>>> 2 CPU sockets of 12 cores, 256GB RAM, 7 HDD 10K, 1 SSD. The idea is
>>> to use GlusterFS to provide HA for the VMs. The 3 servers have a
>>> dual 40Gb NIC and a dual 10Gb NIC. So my intention is to create a
>>> loop like a server triangle using the 40Gb NICs for virtualization
>>> files (VMs .qcow2) access and to move VMs around the pod (east /west
>>> traffic) while using the 10Gb interfaces for giving services to the
>>> outside world (north/south traffic).
>>>
>>>
>>> This said, my first question is: How should I deploy GlusterFS in
>>> such oVirt scenario? My questions are:
>>>
>>>
>>> 1 - Should I create 3 RAID (i.e.: RAID 5), one on each oVirt node,
>>> and then create a GlusterFS using them?
>>>
>>> 2 - Instead, should I create a JBOD array made of all server's disks?
>>>
>>> 3 - What is the best Gluster configuration to provide for HA while
>>> not consuming too much disk space?
>>>
>>> 4 - Does a oVirt hypervisor pod like I am planning to build, and the
>>> virtualization environment, benefits from tiering when using a SSD
>>> disk? And yes, will Gluster do it by default or I have to configure
>>> it to do so?
>>>
>>>
>>> At the bottom line, what is the good practice for using GlusterFS in
>>> small pods for enterprises?
>>>
>>>
>>> You opinion/feedback will be really appreciated!
>>>
>>> Moacir
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170807/f9affb2c/attachment.html>
------------------------------
_______________________________________________
Users mailing list
Users at ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
End of Users Digest, Vol 71, Issue 32
*************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170807/3ad66b1d/attachment-0001.html>
More information about the Users
mailing list