[ovirt-users] Re: Speed Issues

I too struggle with speed issues in hci. Latency is a big problem with writes for me especially when dealing with small file workloads. How are you testing exactly? Look into enabling libgfapi and try some comparisons with that. People have been saying it’s much faster, but it’s not a default option and has a few bugs. Redhat devs do not appear to be giving its implementation any priority unfortunately. I’ve been considering switching to nfs storage because I’m seeing much better performance in testing with it. I have some nvme drives on the way and am curious how they would perform in hci but I’m thinking the issue is not a disk bottleneck (that appears very obvious in your case as well) On Mon, Mar 23, 2020 at 6:44 PM Christian Reiss <email(a)christian-reiss.de&gt; wrote: > Hey folks, > > gluster related question. Having SSD in a RAID that can do 2 GB writes > and Reads (actually above, but meh) in a 3-way HCI cluster connected > with 10gbit connection things are pretty slow inside gluster. > > I have these settings: > > Options Reconfigured: > cluster.server-quorum-type: server > cluster.quorum-type: auto > cluster.shd-max-threads: 8 > features.shard: on > features.shard-block-size: 64MB > server.event-threads: 8 > user.cifs: off > cluster.shd-wait-qlength: 10000 > cluster.locking-scheme: granular > cluster.eager-lock: enable > performance.low-prio-threads: 32 > network.ping-timeout: 30 > cluster.granular-entry-heal: enable > storage.owner-gid: 36 > storage.owner-uid: 36 > cluster.choose-local: true > client.event-threads: 16 > performance.strict-o-direct: on > network.remote-dio: enable > performance.client-io-threads: on > nfs.disable: on > storage.fips-mode-rchecksum: on > transport.address-family: inet > cluster.readdir-optimize: on > cluster.metadata-self-heal: on > cluster.data-self-heal: on > cluster.entry-self-heal: on > cluster.data-self-heal-algorithm: full > features.uss: enable > features.show-snapshot-directory: on > features.barrier: disable > auto-delete: enable > snap-activate-on-create: enable > > Writing inside the /gluster_bricks yields those 2GB/sec writes, Reading > the same. > > Reading inside the /rhev/data-center/mnt/glusterSD/ dir reads go down to > 366mb/sec while writes plummet to to 200mb/sec. > > Summed up: Writing into the SSD Raid in the lvm/xfs gluster brick > directory is fast, writing into the mounted gluster dir is horribly slow. > > The above can be seen and repeated on all 3 servers. The network can do > full 10gbit (tested with, among others: rsync, iperf3). > > Anyone with some idea on whats missing/ going on here? > > Thanks folks, > as always stay safe and healthy! > > -- > with kind regards, > mit freundlichen Gruessen, > > Christian Reiss > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/OMAAERV4IUI... >