On Fri, Jan 21, 2022 at 10:45 AM Gilboa Davara <gilboad@gmail.com> wrote:
On Thu, Jan 20, 2022 at 8:52 AM <jonas@rabe.ch> wrote:
Hi all,

Feel free to give your personal opinion whether software RAID makes any sense at all with oVirt Node or what your architecture would be given the mentioned hardware (3 servers, 6 800GB NVME disks, 18 7.68TB NVME disks).

Thanks a lot for any feedback,
Jonas

Hello,

I somehow missed your original email.
While I don't have a comparable setup (We usually opt for mixed SSD HDD setups with fast VMs running on SSD bached gluster volumes and large-storage VMs using HDD backed gluster volume), I can offer some insight.
1. Performance wise, using software RAID10 on the NVME drives will give you great performance, while reducing the chance of a double drop.
2. Avoid using RAID5/50/6/60 as the write amplification will eat up your read-intensive NVMEs life span in a couple of months.
3, Use a fast network back-end. A good high-end server with hardware RAID HDD can easily saturate a 10GbR Linux @peak load.
4. Use arbiter volumes, split the load between the nodes to reduce the write amplifications. (N1: brick, brick, arbiter. N2: brick, arbiter, brick. N3: arbiter, brick, brick).
5. In general I usually avoid using the wizard, manually creating the gluster volumes before I begin the deployment.

- Gilboa

I missed the fact that your 3'rd host has considerably smaller storage.
In this case, luckily I have a comparable setup @Home. I simply placed all the arbiter volumes on the 3'rd host.

- Gilboa