I have 3 server HCI with Gluster replica 3 storage (10GBe and SSD disks). Small file performance inner-vm is pretty terrible compared to a similar spec'ed VM using NFS mount (10GBe network, SSD disk)

VM with gluster storage:

# dd if=/dev/zero of=test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB) copied, 53.9616 s, 9.5 kB/s

VM with NFS:

# dd if=/dev/zero of=test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB) copied, 2.20059 s, 233 kB/s

This is a very big difference, 2 seconds to copy 1000 files on NFS VM VS 53 seconds on the other.

Aside from enabling libgfapi is there anything I can tune on the gluster or VM side to improve small file performance? I have seen some guides by Redhat in regards to small file performance but I'm not sure what/if any of it applies to oVirt's implementation of gluster in HCI.