What do you mean by "small block sizes"?
inside a VM, or directly on the mounted glusterfs;
dd if=/dev/zero of=tmpfile bs=1M count=100 oflag=direct
of course, a terrible way to write data, but also things like compiling software inside
one of the VMs was terrible slow, 5-10x slower than hardware, consisting of almost only
idling.
Uploading disk images never got above 30MB/s.
(and I did try all options i could find; using upload_disk.py on one of the hosts, even
through a unix socket or with -d option, tweaking buffer size, all of which made no
difference).
Adding an NFS volume and uploading to it I reach +200MB/s.
I tried tuning a few parameters on glusterfs but saw no improvements until I got to
network.remote-dio, which made everything listed above really fast.
Note that network.remote-dio is not the recommended configuration
for ovirt, in particular if on hyperconverge setup when it can be harmful
by delaying sanlock I/O.
https://github.com/oVirt/ovirt-site/blob/4a9b28aac48870343c5ea4d1e83a63c1...
(Patch in discussion)
Oh, I had seen this page, thanks. Is "remote-dio=enabled" harmful as in things
breaking, or just worse performance?
I was a bit reluctant to turn it on, but after seeing it was part of the virt group I
thought it must have been safe.
Perhaps some of the other options like "performance.strict-o-direct on" would
solve my performance issues in a nicer way (I will test it out first thing on monday)
Thanks (I'm not much of a filesystem guy, thanks for putting up with my ignorance)