<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hi Fernando,</p>
<p><br>
</p>
<p>Since I have 3 servers then each one will have a connection to the other two. In this case I could setup 3 subnets, one for each link, avoiding in this way to implement bridging (and Spanning Tree) on the servers. My idea is that when I build the server
I should use the IP addresses of the 40Gb NICs to setup Gluster and everything should to be ok. Anyway, I will test it on a virtual environment before installing.<br>
</p>
<br>
Thanks,<br>
Moacir<br>
<div style="color: rgb(49, 55, 57);"><font size="2"><span style="font-size:10pt;">
<div class="PlainText"><br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Mon, 7 Aug 2017 10:08:32 -0300<br>
From: FERNANDO FREDIANI <fernando.frediani@upx.com><br>
To: users@ovirt.org<br>
Subject: Re: [ovirt-users] Good practices<br>
Message-ID: <8dd168a0-d5ab-bdf4-5c5d-197909afc515@upx.com><br>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"<br>
<br>
Moacir, I beleive for using the 3 servers directly connected to each <br>
others without switch you have to have a Bridge on each server for every <br>
2 physical interfaces to allow the traffic passthrough in layer2 (Is it <br>
possible to create this from the oVirt Engine Web Interface?). If your <br>
ovirtmgmt network is separate from other (should really be) that should <br>
be fine to do.<br>
<br>
<br>
Fernando<br>
<br>
<br>
On 07/08/2017 07:13, Moacir Ferreira wrote:<br>
><br>
> Hi, in-line responses.<br>
><br>
><br>
> Thanks,<br>
><br>
> Moacir<br>
><br>
><br>
> ------------------------------------------------------------------------<br>
> *From:* Yaniv Kaul <ykaul@redhat.com><br>
> *Sent:* Monday, August 7, 2017 7:42 AM<br>
> *To:* Moacir Ferreira<br>
> *Cc:* users@ovirt.org<br>
> *Subject:* Re: [ovirt-users] Good practices<br>
><br>
><br>
> On Sun, Aug 6, 2017 at 5:49 PM, Moacir Ferreira <br>
> <moacirferreira@hotmail.com <<a href="mailto:moacirferreira@hotmail.com">mailto:moacirferreira@hotmail.com</a>>> wrote:<br>
><br>
> I am willing to assemble a oVirt "pod", made of 3 servers, each<br>
> with 2 CPU sockets of 12 cores, 256GB RAM, 7 HDD 10K, 1 SSD. The<br>
> idea is to use GlusterFS to provide HA for the VMs. The 3 servers<br>
> have a dual 40Gb NIC and a dual 10Gb NIC. So my intention is to<br>
> create a loop like a server triangle using the 40Gb NICs for<br>
> virtualization files (VMs .qcow2) access and to move VMs around<br>
> the pod (east /west traffic) while using the 10Gb interfaces for<br>
> giving services to the outside world (north/south traffic).<br>
><br>
><br>
> Very nice gear. How are you planning the network exactly? Without a <br>
> switch, back-to-back? (sounds OK to me, just wanted to ensure this is <br>
> what the 'dual' is used for). However, I'm unsure if you have the <br>
> correct balance between the interface speeds (40g) and the disks (too <br>
> many HDDs?).<br>
><br>
> Moacir:The idea is to have a very high performance network for the <br>
> distributed file system and to prevent bottlenecks when we move one VM <br>
> from a node to another. Using 40Gb NICs I can just connect the servers <br>
> back-to-back. In this case I don't need the expensive 40Gb switch, I <br>
> get very high speed and no contention between north/south traffic with <br>
> east/west.<br>
><br>
><br>
> This said, my first question is: How should I deploy GlusterFS in<br>
> such oVirt scenario? My questions are:<br>
><br>
><br>
> 1 - Should I create 3 RAID (i.e.: RAID 5), one on each oVirt node,<br>
> and then create a GlusterFS using them?<br>
><br>
> I would assume RAID 1 for the operating system (you don't want a <br>
> single point of failure there?) and the rest JBODs. The SSD will be <br>
> used for caching, I reckon? (I personally would add more SSDs instead <br>
> of HDDs, but it does depend on the disk sizes and your space requirements.<br>
><br>
> Moacir: Yes, I agree that I need a RAID-1 for the OS. Now, generic <br>
> JBOD or a JBOD assembled using RAID-5 "disks" createdby the server's <br>
> disk controller?<br>
><br>
> 2 - Instead, should I create a JBOD array made of all server's disks?<br>
><br>
> 3 - What is the best Gluster configuration to provide for HA while<br>
> not consuming too much disk space?<br>
><br>
><br>
> Replica 2 + Arbiter sounds good to me.<br>
> Moacir:I agree, and that is what I am using.<br>
><br>
> 4 - Does a oVirt hypervisor pod like I am planning to build, and<br>
> the virtualization environment, benefits from tiering when using a<br>
> SSD disk? And yes, will Gluster do it by default or I have to<br>
> configure it to do so?<br>
><br>
><br>
> Yes, I believe using lvmcache is the best way to go.<br>
><br>
> Moacir: Are you sure? I say that because the qcow2 files will be<br>
> quite big. So if tiering is "file based" the SSD would have to be<br>
> very, very big unless Gluster tiering do it by "chunks of data".<br>
><br>
><br>
> At the bottom line, what is the good practice for using GlusterFS<br>
> in small pods for enterprises?<br>
><br>
><br>
> Don't forget jumbo frames. libgfapi (coming hopefully in 4.1.5). <br>
> Sharding (enabled out of the box if you use a hyper-converged setup <br>
> via gdeploy).<br>
> *Moacir:* Yes! This is another reason to have separate networks for <br>
> north/south and east/west. In that way I can use the standard MTU on <br>
> the 10Gb NICs and jumbo frames on the file/move 40Gb NICs.<br>
><br>
> Y.<br>
><br>
><br>
> You opinion/feedback will be really appreciated!<br>
><br>
> Moacir<br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> Users@ovirt.org <<a href="mailto:Users@ovirt.org">mailto:Users@ovirt.org</a>><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
> <<a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> Users@ovirt.org<br>
> <a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.ovirt.org/pipermail/users/attachments/20170807/f4181b39/attachment-0001.html">http://lists.ovirt.org/pipermail/users/attachments/20170807/f4181b39/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Mon, 7 Aug 2017 15:26:03 +0200<br>
From: Erekle Magradze <erekle.magradze@recogizer.de><br>
To: FERNANDO FREDIANI <fernando.frediani@upx.com>, users@ovirt.org<br>
Subject: Re: [ovirt-users] Good practices<br>
Message-ID: <aa829d07-fa77-3ed9-2500-e33cc01414b6@recogizer.de><br>
Content-Type: text/plain; charset="utf-8"; Format="flowed"<br>
<br>
Hi Frenando,<br>
<br>
Here is my experience, if you consider a particular hard drive as a <br>
brick for gluster volume and it dies, i.e. it becomes not accessible <br>
it's a huge hassle to discard that brick and exchange with another one, <br>
since gluster some tries to access that broken brick and it's causing <br>
(at least it cause for me) a big pain, therefore it's better to have a <br>
RAID as brick, i.e. have RAID 1 (mirroring) for each brick, in this case <br>
if the disk is down you can easily exchange it and rebuild the RAID <br>
without going offline, i.e switching off the volume doing brick <br>
manipulations and switching it back on.<br>
<br>
Cheers<br>
<br>
Erekle<br>
<br>
<br>
On 08/07/2017 03:04 PM, FERNANDO FREDIANI wrote:<br>
><br>
> For any RAID 5 or 6 configuration I normally follow a simple gold rule <br>
> which gave good results so far:<br>
> - up to 4 disks RAID 5<br>
> - 5 or more disks RAID 6<br>
><br>
> However I didn't really understand well the recommendation to use any <br>
> RAID with GlusterFS. I always thought that GlusteFS likes to work in <br>
> JBOD mode and control the disks (bricks) directlly so you can create <br>
> whatever distribution rule you wish, and if a single disk fails you <br>
> just replace it and which obviously have the data replicated from <br>
> another. The only downside of using in this way is that the <br>
> replication data will be flow accross all servers but that is not much <br>
> a big issue.<br>
><br>
> Anyone can elaborate about Using RAID + GlusterFS and JBOD + GlusterFS.<br>
><br>
> Thanks<br>
> Regards<br>
> Fernando<br>
><br>
><br>
> On 07/08/2017 03:46, Devin Acosta wrote:<br>
>><br>
>> Moacir,<br>
>><br>
>> I have recently installed multiple Red Hat Virtualization hosts for <br>
>> several different companies, and have dealt with the Red Hat Support <br>
>> Team in depth about optimal configuration in regards to setting up <br>
>> GlusterFS most efficiently and I wanted to share with you what I learned.<br>
>><br>
>> In general Red Hat Virtualization team frowns upon using each DISK of <br>
>> the system as just a JBOD, sure there is some protection by having <br>
>> the data replicated, however, the recommendation is to use RAID 6 <br>
>> (preferred) or RAID-5, or at least RAID-1 at the very least.<br>
>><br>
>> Here is the direct quote from Red Hat when I asked about RAID and Bricks:<br>
>> /<br>
>> /<br>
>> /"A typical Gluster configuration would use RAID underneath the <br>
>> bricks. RAID 6 is most typical as it gives you 2 disk failure <br>
>> protection, but RAID 5 could be used too. Once you have the RAIDed <br>
>> bricks, you'd then apply the desired replication on top of that. The <br>
>> most popular way of doing this would be distributed replicated with <br>
>> 2x replication. In general you'll get better performance with larger <br>
>> bricks. 12 drives is often a sweet spot. Another option would be to <br>
>> create a separate tier using all SSD?s.? /<br>
>><br>
>> /In order to SSD tiering from my understanding you would need 1 x <br>
>> NVMe drive in each server, or 4 x SSD hot tier (it needs to be <br>
>> distributed, replicated for the hot tier if not using NVME). So with <br>
>> you only having 1 SSD drive in each server, I?d suggest maybe looking <br>
>> into the NVME option. /<br>
>> /<br>
>> /<br>
>> /Since your using only 3-servers, what I?d probably suggest is to do <br>
>> (2 Replicas + Arbiter Node), this setup actually doesn?t require the <br>
>> 3rd server to have big drives at all as it only stores meta-data <br>
>> about the files and not actually a full copy. /<br>
>> /<br>
>> /<br>
>> /Please see the attached document that was given to me by Red Hat to <br>
>> get more information on this. Hope this information helps you./<br>
>> /<br>
>> /<br>
>><br>
>> --<br>
>><br>
>> Devin Acosta, RHCA, RHVCA<br>
>> Red Hat Certified Architect<br>
>><br>
>> On August 6, 2017 at 7:29:29 PM, Moacir Ferreira <br>
>> (moacirferreira@hotmail.com <<a href="mailto:moacirferreira@hotmail.com">mailto:moacirferreira@hotmail.com</a>>) wrote:<br>
>><br>
>>> I am willing to assemble a oVirt "pod", made of 3 servers, each with <br>
>>> 2 CPU sockets of 12 cores, 256GB RAM, 7 HDD 10K, 1 SSD. The idea is <br>
>>> to use GlusterFS to provide HA for the VMs. The 3 servers have a <br>
>>> dual 40Gb NIC and a dual 10Gb NIC. So my intention is to create a <br>
>>> loop like a server triangle using the 40Gb NICs for virtualization <br>
>>> files (VMs .qcow2) access and to move VMs around the pod (east /west <br>
>>> traffic) while using the 10Gb interfaces for giving services to the <br>
>>> outside world (north/south traffic).<br>
>>><br>
>>><br>
>>> This said, my first question is: How should I deploy GlusterFS in <br>
>>> such oVirt scenario? My questions are:<br>
>>><br>
>>><br>
>>> 1 - Should I create 3 RAID (i.e.: RAID 5), one on each oVirt node, <br>
>>> and then create a GlusterFS using them?<br>
>>><br>
>>> 2 - Instead, should I create a JBOD array made of all server's disks?<br>
>>><br>
>>> 3 - What is the best Gluster configuration to provide for HA while <br>
>>> not consuming too much disk space?<br>
>>><br>
>>> 4 - Does a oVirt hypervisor pod like I am planning to build, and the <br>
>>> virtualization environment, benefits from tiering when using a SSD <br>
>>> disk? And yes, will Gluster do it by default or I have to configure <br>
>>> it to do so?<br>
>>><br>
>>><br>
>>> At the bottom line, what is the good practice for using GlusterFS in <br>
>>> small pods for enterprises?<br>
>>><br>
>>><br>
>>> You opinion/feedback will be really appreciated!<br>
>>><br>
>>> Moacir<br>
>>><br>
>>> _______________________________________________<br>
>>> Users mailing list<br>
>>> Users@ovirt.org <<a href="mailto:Users@ovirt.org">mailto:Users@ovirt.org</a>><br>
>>> <a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Users mailing list<br>
>> Users@ovirt.org<br>
>> <a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> Users@ovirt.org<br>
> <a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.ovirt.org/pipermail/users/attachments/20170807/f9affb2c/attachment.html">http://lists.ovirt.org/pipermail/users/attachments/20170807/f9affb2c/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Users mailing list<br>
Users@ovirt.org<br>
<a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br>
<br>
End of Users Digest, Vol 71, Issue 32<br>
*************************************<br>
</div>
</span></font></div>
</div>
</body>
</html>