I have 3 server HCI with Gluster replica 3 storage (10GBe and SSD disks).
Small file performance inner-vm is pretty terrible compared to a similar
spec'ed VM using NFS mount (10GBe network, SSD disk)
VM with gluster storage:
# dd if=/dev/zero of=test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB) copied, 53.9616 s, 9.5 kB/s
VM with NFS:
# dd if=/dev/zero of=test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB) copied, 2.20059 s, 233 kB/s
This is a very big difference, 2 seconds to copy 1000 files on NFS VM VS 53
seconds on the other.
Aside from enabling libgfapi is there anything I can tune on the gluster or
VM side to improve small file performance? I have seen some guides by
Redhat in regards to small file performance but I'm not sure what/if any of
it applies to oVirt's implementation of gluster in HCI.