7:37 p.m.
Hi,
We are currently building a three node hyper-converged cluster based on oVirt Node and
Gluster. While discussing the different storage layout we couldn't get to a final
decision.
Currently our servers are equipped as follows:
- servers 1 & 2:
- Two 800GB disks for OS
- 100GB RAID1 used as LVM PV for OS
- Nine 7.68TB disks for Gluster
- 60TB RAID 5 used as LVM PV for Gluster
- server 3
- Two 800GB disks for OS & Gluster
- 100GB RAID 1 used as LVM PV for OS
- 700GB RAID 1 used as LVM PV for Gluster
Unfortunately I couldn't find much information about mdadm on this topic. The
hyper-convergence guides ([1], [2]) seem to assume that there is either a hardware RAID in
place or JBOD is used. Is there some documentation available on what to consider when
using mdadm? Or would it be more sensible to just use JBOD and then add redundancy on the
LVM or Gluster level?
If choosing to go with mdadm, what option should I choose in the bricks wizard screen
(RAID 5 or JBOD)?
[1]:
https://ovirt.org/documentation/gluster-hyperconverged/chap-Deploying_Hyp...
[2]:
https://access.redhat.com/documentation/en-us/red_hat_hyperconverged_infr...
7:31 p.m.
Hi all,
Feel free to give your personal opinion whether software RAID makes any sense at all with
oVirt Node or what your architecture would be given the mentioned hardware (3 servers, 6
800GB NVME disks, 18 7.68TB NVME disks).
Thanks a lot for any feedback,
Jonas
11:45 a.m.
On Thu, Jan 20, 2022 at 8:52 AM <jonas(a)rabe.ch> wrote:
Hi all,
Feel free to give your personal opinion whether software RAID makes any
sense at all with oVirt Node or what your architecture would be given the
mentioned hardware (3 servers, 6 800GB NVME disks, 18 7.68TB NVME disks).
Thanks a lot for any feedback,
Jonas
Hello,
I somehow missed your original email.
While I don't have a comparable setup (We usually opt for mixed SSD HDD
setups with fast VMs running on SSD bached gluster volumes and
large-storage VMs using HDD backed gluster volume), I can offer some
insight.
1. Performance wise, using software RAID10 on the NVME drives will give you
great performance, while reducing the chance of a double drop.
2. Avoid using RAID5/50/6/60 as the write amplification will eat up your
read-intensive NVMEs life span in a couple of months.
3, Use a fast network back-end. A good high-end server with hardware RAID
HDD can easily saturate a 10GbR Linux @peak load.
4. Use arbiter volumes, split the load between the nodes to reduce the
write amplifications. (N1: brick, brick, arbiter. N2: brick, arbiter,
brick. N3: arbiter, brick, brick).
5. In general I usually avoid using the wizard, manually creating the
gluster volumes before I begin the deployment.
- Gilboa
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZZ5VDBPHTEV...
11:47 a.m.
On Fri, Jan 21, 2022 at 10:45 AM Gilboa Davara <gilboad(a)gmail.com> wrote:
On Thu, Jan 20, 2022 at 8:52 AM <jonas(a)rabe.ch> wrote:
> Hi all,
>
> Feel free to give your personal opinion whether software RAID makes any
> sense at all with oVirt Node or what your architecture would be given the
> mentioned hardware (3 servers, 6 800GB NVME disks, 18 7.68TB NVME disks).
>
> Thanks a lot for any feedback,
> Jonas
>
Hello,
I somehow missed your original email.
While I don't have a comparable setup (We usually opt for mixed SSD HDD
setups with fast VMs running on SSD bached gluster volumes and
large-storage VMs using HDD backed gluster volume), I can offer some
insight.
1. Performance wise, using software RAID10 on the NVME drives will give
you great performance, while reducing the chance of a double drop.
2. Avoid using RAID5/50/6/60 as the write amplification will eat up your
read-intensive NVMEs life span in a couple of months.
3, Use a fast network back-end. A good high-end server with hardware RAID
HDD can easily saturate a 10GbR Linux @peak load.
4. Use arbiter volumes, split the load between the nodes to reduce the
write amplifications. (N1: brick, brick, arbiter. N2: brick, arbiter,
brick. N3: arbiter, brick, brick).
5. In general I usually avoid using the wizard, manually creating the
gluster volumes before I begin the deployment.
- Gilboa
I missed the fact that your 3'rd host has considerably smaller storage.
In this case, luckily I have a comparable setup @Home. I simply placed all
the arbiter volumes on the 3'rd host.
- Gilboa
4:31 p.m.
Using the wizzard is utilizing the Gluster Andible roles.I would highly recommend using
it, unless you know what you are doing (for example storage alignment when using Hardware
raid).
Keep in mind that the DHT xlator (the logic in distributed volumes) is shard aware, so
your shards are spread between subvolumes and additional performance can be gained.So
using replicated-distributed volumes have their benefits.
If you decide to avoid the software raid, use only replica3 volumes as with SSDs/NVMEs
usually the failures are not physical, but logical (maximum writes reached ->
predictive failure -> total failure).
Also, consider mounting via noatime/relatime and
context="system_u:object_r:glusterd_brick_t:s0" for your gluster bricks.
Best Regards,Strahil Nikolov
On Fri, Jan 21, 2022 at 11:00, Gilboa Davara<gilboad(a)gmail.com> wrote:
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/U2ZEWLRF5D6...
1034
days inactive
1049
days old
4 comments
3 participants
participants (3)
-
Gilboa Davara -
jonas＠rabe.ch -
Strahil Nikolov