According to gluster administration guide:
https://docs.gluster.org/en/latest/Administrator%20Guide/Network%20Config...
in the "when to bond" section we can read:
network throughput limit of client/server \<\< storage throughput limit
1 GbE (almost always)
10-Gbps links or faster -- for writes, replication doubles the load on the network and
replicas are usually on different peers to which the client can transmit in parallel.
So if you are using oVirt hyper-converged in replica 3 you have to transmit everything
two times over the storage network to sync it with other peers.
I'm not really in that details, but
if https://bugzilla.redhat.com/1673058 is really
like it's described, we even have an 5x overhead with current gluster 5.x.
This means that with a 1000 Mbps nic we cannot expect more than:
1000 Mbps / 2 (other replicas) / 5 (overhead in Gluster 5.x ???) / 8 (bit per bytes) =
12.5 MByte per seconds and this is definitively enough to have sanlock failing especially
because we don't have just the sanlock load as you can imagine.
I'd strongly advice to move to 10 Gigabit Ethernet (nowadays with a few hundred
dollars you can buy a 4/5 ports 10GBASE-T copper switch plus 3 nics and the cables just
for the gluster network) or to bond a few 1 Gigabit Ethernet links.
I didn't know that.
So , with 1 Gbit network everyone should use replica 3 arbiter 1 volumes to minimize
replication traffic.
Best Regards,
Strahil Nikolov