Hi,
What is your dirty cache settings on the gluster servers ?
Best Regards,
Strahil NikolovOn Apr 13, 2019 00:44, Alex McWhirter <alex(a)triadic.us>
wrote:
>
> I have 8 machines acting as gluster servers. They each have 12 drives
> raid 50'd together (3 sets of 4 drives raid 5'd then 0'd together as
> one).
>
> They connect to the compute hosts and to each other over lacp'd 10GB
> connections split across two cisco nexus switched with VPC.
>
> Gluster has the following set.
>
> performance.write-behind-window-size: 4MB
> performance.flush-behind: on
> performance.stat-prefetch: on
> server.event-threads: 4
> client.event-threads: 8
> performance.io-thread-count: 32
> network.ping-timeout: 30
> cluster.granular-entry-heal: enable
> performance.strict-o-direct: on
> storage.owner-gid: 36
> storage.owner-uid: 36
> features.shard: on
> cluster.shd-wait-qlength: 10000
> cluster.shd-max-threads: 8
> cluster.locking-scheme: granular
> cluster.data-self-heal-algorithm: full
> cluster.server-quorum-type: server
> cluster.quorum-type: auto
> cluster.eager-lock: enable
> network.remote-dio: off
> performance.low-prio-threads: 32
> performance.io-cache: off
> performance.read-ahead: off
> performance.quick-read: off
> auth.allow: *
> user.cifs: off
> transport.address-family: inet
> nfs.disable: off
> performance.client-io-threads: on
>
>
> I have the following sysctl values on gluster client and servers,
> using
> libgfapi, MTU 9K
>
> net.core.rmem_max = 134217728
> net.core.wmem_max = 134217728
> net.ipv4.tcp_rmem = 4096 87380 134217728
> net.ipv4.tcp_wmem = 4096 65536 134217728
> net.core.netdev_max_backlog = 300000
> net.ipv4.tcp_moderate_rcvbuf =1
> net.ipv4.tcp_no_metrics_save = 1
> net.ipv4.tcp_congestion_control=htcp
>
> reads with this setup are perfect, benchmarked in VM to be about
> 770MB/s
> sequential with disk access times of < 1ms. Writes on the other hand
> are
> all over the place. They peak around 320MB/s sequential write, which
> is
> what i expect but it seems as if there is some blocking going on.
>
> During the write test i will hit 320MB/s briefly, then 0MB/s as disk
> access time shoot to over 3000ms, then back to 320MB/s. It averages
> out
> to about 110MB/s afterwards.
>
> Gluster version is 3.12.15 ovirt is 4.2.7.5
>
> Any ideas on what i could tune to eliminate or minimize that blocking?
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
>
https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z7F72BKYKAG...
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FMB6NCNJL2W...
Just the vdsm defaults
vm.dirty_ratio = 5
vm.dirty_background_ratio = 2
these boxes only have 8gb of ram as well, so those percentages should be
super small.