4:03 a.m.
Hi,
Can someone please give me some pointers, what would be the best setup for
performance and reliability?
We have the following hardware setup:
3x Supermicro server with following features per server:
128GB RAM
4x 8TB SATA HDD
2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Quad port 10Gbe Inter NIC
2x 10GB Cisco switches (to isolate storage network from LAN)
One of the servers will be in another office, with a 600Mb wireless link
for Disaster Recovery.
What is recommended for the best setup in terms of redundancy and speed?
I am guessing GlusterFS with a Distributed Striped Replicated Volume across
3 of the servers.
For added performance I want to use the SSD drives, perhaps with dm-cache?
Should I combine the 4x HDD's using LVM on each host node?
What about RAID 6?
Virtual Machines will then reside on the oVirt Cluster and any one of the 3
host nodes can fail, or any single HDD can fail and all should still work,
right/?
--
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za
5:47 a.m.
This is a multi-part message in MIME format.
--------------714A7C973919A95EE2A8FDD0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Helli Rudi
Nice specs.
I wouldn't use GlusterFS for this setup with the third server in a
different location. Just have this server as an Standalone and replicate
the VMs there. You won't have real time replication, but much less
hassle and probably to have constant failures, specially knowing you
have a wireless link.
For the SSDs I have been using bcache with success. Relatively simple to
setup and pretty good performance.
For your specs as you have 4 mechanical disks I would recommend you to
have a RAID 5 between them (4 disks is my limit for RAID 5) and a RAID 0
made of SSDs for the bcache device. If the RAID 0 fails for any reason
it will fall back directly to the mechanical disks and you can do
maintenance on the Node doing live migration in order to replace the
failed disks.
However as you have you have 2 remaining server to create your cluster
you may need to consider GlusterFS on the top of this RAID to have the
replication and Highavaibility.
Hope it helps.
Fernando
On 13/11/2017 08:03, Rudi Ahlers wrote:
--------------714A7C973919A95EE2A8FDD0
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Helli Rudi</p>
<p>Nice specs.</p>
<p>I wouldn't use GlusterFS for this setup with the third server in
a different location. Just have this server as an Standalone and
replicate the VMs there. You won't have real time replication, but
much less hassle and probably to have constant failures, specially
knowing you have a wireless link.</p>
<p>For the SSDs I have been using bcache with success. Relatively
simple to setup and pretty good performance.</p>
<p>For your specs as you have 4 mechanical disks I would recommend
you to have a RAID 5 between them (4 disks is my limit for RAID 5)
and a RAID 0 made of SSDs for the bcache device. If the RAID 0
fails for any reason it will fall back directly to the mechanical
disks and you can do maintenance on the Node doing live migration
in order to replace the failed disks.</p>
<p>However as you have you have 2 remaining server to create your
cluster you may need to consider GlusterFS on the top of this RAID
to have the replication and Highavaibility.</p>
<p>Hope it helps.</p>
<p>Fernando<br>
</p>
<br>
<div class="moz-cite-prefix">On 13/11/2017 08:03, Rudi Ahlers
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAAj3DjmsZOkkzTZ_wYz-B7_xV9YoKgknyD0r=RJu5=sa0i7=wg@mail.gmail.com">
<div dir="ltr">Hi,
<div><br>
</div>
<div>Can someone please give me some pointers, what would be the
best setup for performance and reliability?</div>
<div><br>
</div>
<div>We have the following hardware setup:</div>
<div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span
style="font-size:14.6667px"><br>
</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">3x
Supermicro server with following features per
server:</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">128GB
RAM</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">4x 8TB
SATA
HDD</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">2x
SSD
drives (intel_ssdsc2ba400g4 - 400GB DC
S3710)</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">2x 12
core
CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @
2.20GHz</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">Quad
port
10Gbe Inter NIC</span></font></div>
<div><font color="#2c2c2c" face="Open Sans, Helvetica,
Arial,
sans-serif"><span style="font-size:14.6667px">2x
10GB
Cisco switches (to isolate storage network from
LAN)</span></font></div>
</div>
<div><br>
</div>
<div>
<div>One of the servers will be in another office, with a
600Mb wireless link for Disaster Recovery. </div>
<div><br>
</div>
<div>What is recommended for the best setup in terms of
redundancy and speed?</div>
<div><br>
</div>
<div>I am guessing GlusterFS with a Distributed Striped
Replicated Volume across 3 of the servers. </div>
<div><br>
</div>
<div>For added performance I want to use the SSD drives,
perhaps with dm-cache?</div>
<div><br>
</div>
<div>Should I combine the 4x HDD's using LVM on each host
node?</div>
<div>What about RAID 6?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>Virtual Machines will then reside on the oVirt Cluster
and any one of the 3 host nodes can fail, or any single
HDD can fail and all should still work, right/? </div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
-- <br>
<div class="gmail_signature">Kind Regards<br>
Rudi Ahlers<br>
Website: <a href="http://www.rudiahlers.co.za"
target="_blank"
moz-do-not-send="true">http://www.rudiahlers.co.za</a><br>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
</body>
</html>
--------------714A7C973919A95EE2A8FDD0--
6:54 a.m.
This is a multi-part message in MIME format.
--------------20F62C253AB9FA2BFC1BDEA7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Just curious. Would Gluster with a JBOD configuration make sense in this
scenario? Maybe with dm-cache on the SSDs?
In general, does a Gluster JBOD configuration make sense as a backend
for oVirt?
Eduardo Mayoral Jimeno (emayoral(a)arsys.es)
Administrador de sistemas. Departamento de Plataformas. Arsys internet.
+34 941 620 145 ext. 5153
On 13/11/17 12:47, FERNANDO FREDIANI wrote:
--------------20F62C253AB9FA2BFC1BDEA7
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Just curious. Would Gluster with a JBOD configuration make sense
in this scenario? Maybe with dm-cache on the SSDs?</p>
<p>In general, does a Gluster JBOD configuration make sense as a
backend for oVirt?<br>
</p>
<pre class="moz-signature" cols="72">Eduardo Mayoral Jimeno
(<a class="moz-txt-link-abbreviated"
href="mailto:emayoral@arsys.es">emayoral@arsys.es</a>)
Administrador de sistemas. Departamento de Plataformas. Arsys internet.
+34 941 620 145 ext. 5153</pre>
<div class="moz-cite-prefix">On 13/11/17 12:47, FERNANDO FREDIANI
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:8940c496-6d8d-ddf7-7b57-3f828f9102d2@upx.com">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
<p>Helli Rudi</p>
<p>Nice specs.</p>
<p>I wouldn't use GlusterFS for this setup with the third server
in a different location. Just have this server as an Standalone
and replicate the VMs there. You won't have real time
replication, but much less hassle and probably to have constant
failures, specially knowing you have a wireless link.</p>
<p>For the SSDs I have been using bcache with success. Relatively
simple to setup and pretty good performance.</p>
<p>For your specs as you have 4 mechanical disks I would recommend
you to have a RAID 5 between them (4 disks is my limit for RAID
5) and a RAID 0 made of SSDs for the bcache device. If the RAID
0 fails for any reason it will fall back directly to the
mechanical disks and you can do maintenance on the Node doing
live migration in order to replace the failed disks.</p>
<p>However as you have you have 2 remaining server to create your
cluster you may need to consider GlusterFS on the top of this
RAID to have the replication and Highavaibility.</p>
<p>Hope it helps.</p>
<p>Fernando<br>
</p>
<br>
<div class="moz-cite-prefix">On 13/11/2017 08:03, Rudi Ahlers
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAAj3DjmsZOkkzTZ_wYz-B7_xV9YoKgknyD0r=RJu5=sa0i7=wg@mail.gmail.com">
<div dir="ltr">Hi,
<div><br>
</div>
<div>Can someone please give me some pointers, what would be
the best setup for performance and reliability?</div>
<div><br>
</div>
<div>We have the following hardware setup:</div>
<div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px"><br>
</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">3x
Supermicro server with following features per
server:</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">128GB
RAM</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">4x
8TB SATA HDD</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">2x
SSD drives (intel_ssdsc2ba400g4 - 400GB DC
S3710)</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">2x
12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @
2.20GHz</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">Quad
port 10Gbe Inter NIC</span></font></div>
<div><font color="#2c2c2c" face="Open Sans,
Helvetica,
Arial, sans-serif"><span
style="font-size:14.6667px">2x
10GB Cisco switches (to isolate storage network from
LAN)</span></font></div>
</div>
<div><br>
</div>
<div>
<div>One of the servers will be in another office, with a
600Mb wireless link for Disaster Recovery. </div>
<div><br>
</div>
<div>What is recommended for the best setup in terms of
redundancy and speed?</div>
<div><br>
</div>
<div>I am guessing GlusterFS with a Distributed Striped
Replicated Volume across 3 of the servers. </div>
<div><br>
</div>
<div>For added performance I want to use the SSD drives,
perhaps with dm-cache?</div>
<div><br>
</div>
<div>Should I combine the 4x HDD's using LVM on each host
node?</div>
<div>What about RAID 6?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>Virtual Machines will then reside on the oVirt
Cluster and any one of the 3 host nodes can fail, or any
single HDD can fail and all should still work, right/? </div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
-- <br>
<div class="gmail_signature">Kind Regards<br>
Rudi Ahlers<br>
Website: <a href="http://www.rudiahlers.co.za"
target="_blank"
moz-do-not-send="true">http://www.rudiahlers.co.za</a><br>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org"
moz-do-not-send="true">Users(a)ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users"
moz-do-not-send="true">http://lists.ovirt.org/mailman/listin...
</pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
</body>
</html>
--------------20F62C253AB9FA2BFC1BDEA7--
2:13 a.m.
On Mon, Nov 13, 2017 at 3:33 PM, Rudi Ahlers <rudiahlers(a)gmail.com> wrote:
All 3 servers need to be on the same network with low latency between
servers. Gluster volume is used in replica 3 configuration in oVirt, and
data will be written to all 3 servers synchronously. If you have one of the
3 servers in a remote location, your writes will be slow and the storage
will seem to be unavailable to oVirt.
For Disaster recovery, you can use gluster's geo-replication feature. This
does need an additional server in the remote location. See
https://www.ovirt.org/develop/release-management/features/gluster/gluster...
Replica 3 gluster volume with sharding turned on for the gluster volume. If
you use the deployment via Cockpit option, the volume is created with all
recommended options.
You can also manually set this by enabling the virt profile, like this
# gluster volume set <volumename> group virt
Should I combine the 4x HDD's using LVM on each host node?
What about RAID 6?
Gluster can work with either RAID or JBOD ( with replica 3, JBOD can be
used, as gluster has the redundancy built in on the other nodes). Users
often choose RAID to aggregate capacity from multiple disks as single
brick, and also as the brick rebuilt time is offloaded to RAID layer in
case of any of the hard disk failures. The answer is it depends on your
needs and the hardware you have.
That's correct. With a replica 3 approach, the oVirt environment is
available even when one of the 3 hosts fail.
2729
days inactive
2730
days old
3 comments
4 participants
participants (4)
-
Eduardo Mayoral -
FERNANDO FREDIANI -
Rudi Ahlers -
Sahina Bose