[ovirt-users] Re: Tuning Gluster Writes
On 2019-04-15 12:58, Alex McWhirter wrote:
On 2019-04-15 12:43, Darrell Budic wrote: Interesting. Who's 10g
cards and which offload settings did you disable? Did you do that on the servers or the vm
host clients or both?
On Apr 15, 2019, at 11:37 AM, Alex McWhirter <alex(a)triadic.us> wrote:
I went in and disabled TCP offload on all the nics, huge performance boost. went from
110MB/s to 240MB/s seq writes, reads lost a bit of performance going down to 680MB/s, but
that's a decent trade off. Latency is still really high though, need to work on that.
I think some more TCP tuning might help.
Those changes didn't do a whole lot, but i ended up enabling performance.read-ahead
on the gluster volume. my blockdev read ahead values were already 8192, which seemed good
enough. Not sure if ovirt set those, or if it's just the defaults of my raid
controller.
Anyways up to 350MB/s writes, 700MB/s reads. Which so happens to correlate with the
saturation of my 10G network. Latency is still a slight issue, but at least now im not
blocking :)
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/5COPHAIVCVK...
These are dual port qlogic QLGE cards, plugging into dual Cisco Nexus
3064's with VPC to allow me to LACP across two switches. These are
FCoE/10GBE cards, so on the Cisco Switches i had to disable lldp on the
ports to stop FCoE initiator errors from disabling ports (as i don't use
FCoE atm)
bond options are "mode=4 lacp_rate=1 miimon=100 xmit_hash_policy=1"
then i have the following /sbin/ifup-local script that triggers on
storage network creation
#!/bin/bash
case "$1" in
Storage)
/sbin/ethtool -K ens2f0 tx off rx off tso off gso off
/sbin/ethtool -K ens2f1 tx off rx off tso off gso off
/sbin/ip link set dev ens2f0 txqueuelen 10000
/sbin/ip link set dev ens2f1 txqueuelen 10000
/sbin/ip link set dev bond2 txqueuelen 10000
/sbin/ip link set dev Storage txqueuelen 10000
;;
*)
;;
esac
exit 0
if you have lro, disable it too IMO, these cards do not do lro so it's
not applicable to me.
This did cut down my read performance by about 50MB/s, but my write went
from 98-110MB/s to about 240MB/s, then enabling read-ahead got me to the
350MB/s it should have been.
Oh and i did it on both, the VM hosts and storage machines. Same cards
in all of them.
Attachments:
- attachment.html (text/html — 4.0 KB)