This is a multi-part message in MIME format. --------------6C7C95E00858D75DD80BDF80 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hello Rudi If you have a 4th server it may work,but I am not knowledgeable about Gluster's GEOreplication. Perhaps someone else can advise. With regards RAID 5 for 4 disks this is intended for capacity and as mentioned 4 disks is the maximum I would use for RAID5. As you have SSDs and intend to use a caching technique like bcache ou dm-cache this should cover the performance hit of RAID 5 write. If you consider using RAID 6 in only 4 disks you are much better of using RAID 10 then as you will have double write performance and same capacity. I normally use RAID 10 or in ZFS configurations (not this case) multiple vdevs of RAID6. For RAID in Linux I have been using mdraid. I know LVM does some Raid but I personally never done myself so can't advise. Regards. Fernando On 13/11/2017 10:19, Rudi Ahlers wrote:

Hi Fernando,

Thanx.

I meant to say, the 4th server will be in another office. It's about 3Km away and I was thinking of using Gluster's GEOreplication for this purpose.

I am not a fond user of RAID5 at all. But this raises the question: does RAID add any unnecessary overhead? I would rather run RAID 6 or RAI10. And then, if RAID is the preferred way (over LVM?), as I don't have dedicated hardware RAID cards, would mdraid add any benefit?

On Mon, Nov 13, 2017 at 1:47 PM, FERNANDO FREDIANI <fernando.frediani@upx.com <mailto:fernando.frediani@upx.com>> wrote:

Helli Rudi

Nice specs.

I wouldn't use GlusterFS for this setup with the third server in a different location. Just have this server as an Standalone and replicate the VMs there. You won't have real time replication, but much less hassle and probably to have constant failures, specially knowing you have a wireless link.

For the SSDs I have been using bcache with success. Relatively simple to setup and pretty good performance.

For your specs as you have 4 mechanical disks I would recommend you to have a RAID 5 between them (4 disks is my limit for RAID 5) and a RAID 0 made of SSDs for the bcache device. If the RAID 0 fails for any reason it will fall back directly to the mechanical disks and you can do maintenance on the Node doing live migration in order to replace the failed disks.

However as you have you have 2 remaining server to create your cluster you may need to consider GlusterFS on the top of this RAID to have the replication and Highavaibility.

Hope it helps.

Fernando

On 13/11/2017 08:03, Rudi Ahlers wrote:

...

Hi,

Can someone please give me some pointers, what would be the best setup for performance and reliability?

We have the following hardware setup:

3x Supermicro server with following features per server: 128GB RAM 4x 8TB SATA HDD 2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710) 2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz Quad port 10Gbe Inter NIC 2x 10GB Cisco switches (to isolate storage network from LAN)

One of the servers will be in another office, with a 600Mb wireless link for Disaster Recovery.

What is recommended for the best setup in terms of redundancy and speed?

I am guessing GlusterFS with a Distributed Striped Replicated Volume across 3 of the servers.

For added performance I want to use the SSD drives, perhaps with dm-cache?

Should I combine the 4x HDD's using LVM on each host node? What about RAID 6?

Virtual Machines will then reside on the oVirt Cluster and any one of the 3 host nodes can fail, or any single HDD can fail and all should still work, right/?

-- Kind Regards Rudi Ahlers Website: http://www.rudiahlers.co.za

_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>

_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>

-- Kind Regards Rudi Ahlers Website: http://www.rudiahlers.co.za

--------------6C7C95E00858D75DD80BDF80 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hello Rudi</p> <p>If you have a 4th server it may work,but I am not knowledgeable about Gluster's GEOreplication. Perhaps someone else can advise.</p> <p>With regards RAID 5 for 4 disks this is intended for capacity and as mentioned 4 disks is the maximum I would use for RAID5. As you have SSDs and intend to use a caching technique like bcache ou dm-cache this should cover the performance hit of RAID 5 write.<br> If you consider using RAID 6 in only 4 disks you are much better of using RAID 10 then as you will have double write performance and same capacity. I normally use RAID 10 or in ZFS configurations (not this case) multiple vdevs of RAID6.<br> </p> <p>For RAID in Linux I have been using mdraid. I know LVM does some Raid but I personally never done myself so can't advise.</p> <p>Regards.<br> Fernando<br> </p> <br> <div class="moz-cite-prefix">On 13/11/2017 10:19, Rudi Ahlers wrote:<br> </div> <blockquote type="cite" cite="mid:CAAj3DjmTaxR5JEESLTVtqyzk4H_cN6StKVB_JNXBjJtWrDJXEw@mail.gmail.com"> <div dir="ltr">Hi Fernando,  <div><br> </div> <div>Thanx. </div> <div><br> </div> <div>I meant to say, the 4th server will be in another office. It's about 3Km away and I was thinking of using Gluster's GEOreplication for this purpose. </div> <div><br> </div> <div>I am not a fond user of RAID5 at all. But this raises the question: does RAID add any unnecessary overhead? I would rather run RAID 6 or RAI10. </div> <div>And then, if RAID is the preferred way (over LVM?), as I don't have dedicated hardware RAID cards, would mdraid add any benefit?</div> </div> <div class="gmail_extra"><br> <div class="gmail_quote">On Mon, Nov 13, 2017 at 1:47 PM, FERNANDO FREDIANI <span dir="ltr"><<a href="mailto:fernando.frediani@upx.com" target="_blank" moz-do-not-send="true">fernando.frediani@upx.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> <p>Helli Rudi</p> <p>Nice specs.</p> <p>I wouldn't use GlusterFS for this setup with the third server in a different location. Just have this server as an Standalone and replicate the VMs there. You won't have real time replication, but much less hassle and probably to have constant failures, specially knowing you have a wireless link.</p> <p>For the SSDs I have been using bcache with success. Relatively simple to setup and pretty good performance.</p> <p>For your specs as you have 4 mechanical disks I would recommend you to have a RAID 5 between them (4 disks is my limit for RAID 5) and a RAID 0 made of SSDs for the bcache device. If the RAID 0 fails for any reason it will fall back directly to the mechanical disks and you can do maintenance on the Node doing live migration in order to replace the failed disks.</p> <p>However as you have you have 2 remaining server to create your cluster you may need to consider GlusterFS on the top of this RAID to have the replication and Highavaibility.</p> <p>Hope it helps.</p> <p>Fernando<br> </p> <div> <div class="h5"> <br> <div class="m_-8722418027031101913moz-cite-prefix">On 13/11/2017 08:03, Rudi Ahlers wrote:<br> </div> </div> </div> <blockquote type="cite"> <div> <div class="h5"> <div dir="ltr">Hi,  <div><br> </div> <div>Can someone please give me some pointers, what would be the best setup for performance and reliability?</div> <div><br> </div> <div>We have the following hardware setup:</div> <div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px"><br> </span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">3x Supermicro server with following features per server:</span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">128GB RAM</span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">4x 8TB SATA HDD</span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)</span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz</span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">Quad port 10Gbe Inter NIC</span></font></div> <div><font color="#2c2c2c" face="Open Sans, Helvetica, Arial, sans-serif"><span style="font-size:14.6667px">2x 10GB Cisco switches (to isolate storage network from LAN)</span></font></div> </div> <div><br> </div> <div> <div>One of the servers will be in another office, with a 600Mb wireless link for Disaster Recovery. </div> <div><br> </div> <div>What is recommended for the best setup in terms of redundancy and speed?</div> <div><br> </div> <div>I am guessing GlusterFS with a Distributed Striped Replicated Volume across 3 of the servers. </div> <div><br> </div> <div>For added performance I want to use the SSD drives, perhaps with dm-cache?</div> <div><br> </div> <div>Should I combine the 4x HDD's using LVM on each host node?</div> <div>What about RAID 6?</div> <div><br> </div> <div><br> </div> <div><br> </div> <div> <div>Virtual Machines will then reside on the oVirt Cluster and any one of the 3 host nodes can fail, or any single HDD can fail and all should still work, right/? </div> </div> <div><br> </div> <div><br> </div> <div><br> </div> <div><br> </div> -- <br> <div class="m_-8722418027031101913gmail_signature">Kind Regards<br> Rudi Ahlers<br> Website: <a href="http://www.rudiahlers.co.za" target="_blank" moz-do-not-send="true">http://www.rudiahlers.co.za</a><br> </div> </div> </div> <br> <fieldset class="m_-8722418027031101913mimeAttachmentHeader"></fieldset> <br> </div> </div> <pre>______________________________<wbr>_________________ Users mailing list <a class="m_-8722418027031101913moz-txt-link-abbreviated" href="mailto:Users@ovirt.org" target="_blank" moz-do-not-send="true">Users@ovirt.org</a> <a class="m_-8722418027031101913moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a> </pre> </blockquote> <br> </div> <br> ______________________________<wbr>_________________<br> Users mailing list<br> <a href="mailto:Users@ovirt.org" moz-do-not-send="true">Users@ovirt.org</a><br> <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br> <br> </blockquote> </div> <br> <br clear="all"> <div><br> </div> -- <br> <div class="gmail_signature" data-smartmail="gmail_signature">Kind Regards<br> Rudi Ahlers<br> Website: <a href="http://www.rudiahlers.co.za" target="_blank" moz-do-not-send="true">http://www.rudiahlers.co.za</a><br> </div> </div> </blockquote> <br> </body> </html> --------------6C7C95E00858D75DD80BDF80--