Perhaps it will help future sysadmins will learn from my mistake.
I also saw very poor upload speeds (~30MB/s) no matter what I tried. I went through the
whole route with unix-sockets and whatnot.
But, in the end, it just turned out that the glusterfs itself was the bottleneck; abysmal
performance for small block sizes.
I found the list of suggested performance tweaks that RHEL suggests. In particular, it was
the "network.remote-dio=on" setting that made all the difference. Almost 10x
faster.
https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.1...