>How much of a discussion will we have, when I say that I want a Gluster volume to expand/grow/shrink and transform from 1 to N bricks and transition between replicas, dispersed volumes, sharded or non sharded with oVirt seamlessly running on top?
It's dead simple -> replica 3 (and arbitraded replica 3) is the safest and the most performant volume. Dispersed volumes are like erasure coding in CEPH. You can save space, but performance will be mediocre.
Yet, oVirt UI prevents you to create and use such volume - but it doesn't prevent you from using such.
In both cases (replicated and dispersed volumes) you cannot expand to an arbitrary brick count.
What you are asking is a distributed volume in gluster, but in order to be safe you need distributed-replicated one.
Imagine that you have 1 brick and you want to add 4 more , so 5 in total.You got 2 options:
A) use distributed volume , but when a node is rebooted you loose access to it's data -> quite not good, right
B) you use distributed-replicated (replica 3 arbiter 1) volume to spread data among 4 nodes and the 5th is your arbiter for both subvolumes
The second option protects you against failures during reboots, yet the upgrade path (safe one) is limited (assuming that you want only 1 brick per host, as you use HW raid and all disks into it) :
1 host -> 3 hosts -> 5/6 hosts (5-> arbitraded, 6-> full replica) and so on.
It's easy to say that you want to go from 1 to N bricks, but safety is always first. Yet, oVirt is not like other proprietary software - so it let's you use POSIX-compliant solutions which allows you to use even a pure distributed volume. It just won't let you shoot your leg with oVirt....
Best Regards,
Strahil Nikolov