Here is the post link: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/I62VBDYPQIWPRE3... Here is the actual link: https://docs.gluster.org/en/latest/Administrator%20Guide/Linux%20Kernel%20Tu... Eric Evans Digital Data Services LLC. 304.660.9080 From: suporte@logicworks.pt <suporte@logicworks.pt> Sent: Wednesday, October 21, 2020 11:47 AM To: eevans@digitaldatatechs.com Cc: users <users@ovirt.org> Subject: Re: [ovirt-users] Improve glusterfs performance Thanks. This is what I found from you related to glusterfs: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DN47OTYUUCOAFD... But I don't find anything on how to imporve gluster. José _____ De: eevans@digitaldatatechs.com <mailto:eevans@digitaldatatechs.com> Para: suporte@logicworks.pt <mailto:suporte@logicworks.pt> , "users" <users@ovirt.org <mailto:users@ovirt.org> > Enviadas: Quarta-feira, 21 De Outubro de 2020 15:53:00 Assunto: RE: [ovirt-users] Improve glusterfs performance I posted a link in the users list that details how to improve gluster and improve performance. Search gluster. Eric Evans Digital Data Services LLC. 304.660.9080 From: suporte@logicworks.pt <mailto:suporte@logicworks.pt> <suporte@logicworks.pt <mailto:suporte@logicworks.pt> > Sent: Wednesday, October 21, 2020 9:42 AM To: users <users@ovirt.org <mailto:users@ovirt.org> > Subject: [ovirt-users] Improve glusterfs performance Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- _____ Jose Ferradeira http://www.logicworks.pt

I agree with Alex. Also, most of the kernel tunables proposed in that thread are also available in the tuned profiles provided by the redhat-storage-server source rpm available at ftp://ftp.redhat.com/redhat/linux/enterprise/7Server/en/RHS/SRPMS/ Usually the alignment of XFS ontop the HW raid is quite important and missed. Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 22:35:21 Гринуич+3, Alex McWhirter <alex@triadic.us> написа: In my experience, the ovirt optimized defaults are fairly sane. I may change a few things like enabling read ahead or increasing the shard size, but these are minor performance bumps if anything. The most important thing is the underlying storage, RAID 10 is ideal performance wise, large stripe sizes are preferable for VM workloads, etc... You want the underlying storage to be as fast as possible, dedicated cache devices are a plus. IOPS and latency are often more important that throughput. Network throughput and latency are also very important. I don't think i would attempt a gluster setup on anything slower than 10GbE, jumbo frames are a huge help, switches with large buffers are nice as well. Do not L3 route gluster (at least not inter-server links) unless you have a switch that can do line rate routing. High or inconsistent network latency will bring gluster to it's knees. Kernel tuning / gluster volume options do help, but they are not groundbreaking performance improvements. Usually just little speed boosts here and there. On 2020-10-21 12:30, eevans@digitaldatatechs.com wrote:

  

...

    

            Here is the post link:

https://lists.ovirt.org/archives/list/users@ovirt.org/thread/I62VBDYPQIWPRE3...

 

Here is the actual link:

https://docs.gluster.org/en/latest/Administrator%20Guide/Linux%20Kernel%20Tu...

 

 

   Eric Evans

Digital Data Services LLC.

304.660.9080

 

      From: suporte@logicworks.pt <suporte@logicworks.pt> Sent: Wednesday, October 21, 2020 11:47 AM To: eevans@digitaldatatechs.com Cc: users <users@ovirt.org> Subject: Re: [ovirt-users] Improve glusterfs performance

 

      Thanks.

   This is what I found from you related to glusterfs: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DN47OTYUUCOAFD...

    

   But I don't find anything on how to imporve gluster.

    

   José

    

________________________________

   De: eevans@digitaldatatechs.comPara: suporte@logicworks.pt, "users" <users@ovirt.org>Enviadas: Quarta-feira, 21 De Outubro de 2020 15:53:00Assunto: RE: [ovirt-users] Improve glusterfs performance

    

      I posted a link in the users list that details how to improve gluster and improve performance. Search gluster.

 

   Eric Evans

Digital Data Services LLC.

304.660.9080

 

      From: suporte@logicworks.pt <suporte@logicworks.pt> Sent: Wednesday, October 21, 2020 9:42 AMTo: users <users@ovirt.org>Subject: [ovirt-users] Improve glusterfs performance

 

      Hello,

    

   Can anyone help me in how can I improve the performance of glusterfs to work with oVirt?

    

   Thanks

    

   --

   ________________________________

   Jose Ferradeirahttp://www.logicworks.pt

 

_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/EAWWGYZ4GUNHDB...

_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LI6QSA4N3JPXXV...

Virt settings are those: [root@ovirt1 slow]# cat /var/lib/glusterd/groups/virt performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable cluster.eager-lock=enable cluster.quorum-type=auto cluster.server-quorum-type=server cluster.data-self-heal-algorithm=full cluster.locking-scheme=granular cluster.shd-max-threads=8 cluster.shd-wait-qlength=10000 features.shard=on user.cifs=off cluster.choose-local=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on I am using cluster.choose-local set to on ,as my Network is far slower than the bricks. Another optimization you can consider is  to test with different values of event-threads , shd-max-threads and maybe low-prio-threads - but this is valid for Fast SSDs/NVMEs. Of course, it's reasonable to have a production environment that is separate from test :) Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 14:00:52 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU... _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4Q7GVB4K2WK4J2...

The engine version is 4.3.9.4-1.el7 I have 2 simple glusterfs, a volume with one brick gfs1 with an old version of gluster (3.7.6) and a volume with one brick gfs2 with version 6.8 2 hosts, node3 is up to date, node 2 is not up to date: RHEL - 7 – 7.1908.0.el7.centos CentOS Linux 7 (Core) 3.10.0 – 1062.18.1.el7.x86_64 2.12.0 – 33.1.el7_7.4 libvirt-4.5.0-23.el7_7.6 vdsm-4.30.43-1.el7 0.14.0 – 7.el7 glusterfs-6.8-1.el7 librbd1-10.2.5-4.el7 openvswitch-2.11.0-4.el7 gfs1 has 3 nic's gigabit in mode 802.3ad gfs2 has 4 nic's gigabit in mode 802.3ad A VM performes better in node3/gfs1: # dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 registos dentro 1+0 registos fora 1073741824 bytes (1,1 GB) copiados, 12,3365 s, 87,0 MB/s in node2/gfs2: dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 51.0159 s, 21.0 MB/s For windows machine is the same. It helps if you have windows drives up to date. De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Domingo, 25 De Outubro de 2020 19:58:12 Assunto: Re: [ovirt-users] Improve glusterfs performance It seems that your e-mail went to the spam. I would start with isolating the issue ? 1. Is this a VM specific issue or a more-wide issue. - Can you move another VM on that storage domain and verify performance ? - Can you create/migrate same OS-type VM and check performance? - What about running a VM with different version of Windows or even better -> Linux ? Also I would check the systems from bottom up. Check the following: - Are you using Latest Firmware/OS updates for the Hosts and Engine ? - What is your cmdline : cat /proc/cmdline - Tuned profile - Are you using a HW controller for the Gluster bricks ? Health of controller and disks ? - Is the FS aligned properly to the HW controller (stripe unit and stripe width )? More details on https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/ht... - FS Fragmentation ? Is your FS full ? - Gluster volume status ? Client count ? (for example: gluster volume status all client-list) - Gluster version, cluster.op-version and cluster.max-op-version ? - If your network is slower than the bricks' backend (disks faster than network) , you can set cluster.choose-local to 'yes' - Any errors and warnings in the gluster logs ? Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 13:59:04 Гринуич+3, <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU...

Yes, the VM's disks are on the distributed volume They were created like this: gluster volume create data transport tcp gfs2.domain.com:/home/brick1 De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 15:53:57 Assunto: Re: [ovirt-users] Improve glusterfs performance Are the VM's disks located on the distributed Volume ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 17:21:49 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: The engine version is 4.3.9.4-1.el7 I have 2 simple glusterfs, a volume with one brick gfs1 with an old version of gluster (3.7.6) and a volume with one brick gfs2 with version 6.8 2 hosts, node3 is up to date, node 2 is not up to date: RHEL - 7 – 7.1908.0.el7.centos CentOS Linux 7 (Core) 3.10.0 – 1062.18.1.el7.x86_64 2.12.0 – 33.1.el7_7.4 libvirt-4.5.0-23.el7_7.6 vdsm-4.30.43-1.el7 0.14.0 – 7.el7 glusterfs-6.8-1.el7 librbd1-10.2.5-4.el7 openvswitch-2.11.0-4.el7 gfs1 has 3 nic's gigabit in mode 802.3ad gfs2 has 4 nic's gigabit in mode 802.3ad A VM performes better in node3/gfs1: # dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 registos dentro 1+0 registos fora 1073741824 bytes (1,1 GB) copiados, 12,3365 s, 87,0 MB/s in node2/gfs2: dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 51.0159 s, 21.0 MB/s For windows machine is the same. It helps if you have windows drives up to date. ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Domingo, 25 De Outubro de 2020 19:58:12 Assunto: Re: [ovirt-users] Improve glusterfs performance It seems that your e-mail went to the spam. I would start with isolating the issue ? 1. Is this a VM specific issue or a more-wide issue. - Can you move another VM on that storage domain and verify performance ? - Can you create/migrate same OS-type VM and check performance? - What about running a VM with different version of Windows or even better -> Linux ? Also I would check the systems from bottom up. Check the following: - Are you using Latest Firmware/OS updates for the Hosts and Engine ? - What is your cmdline : cat /proc/cmdline - Tuned profile - Are you using a HW controller for the Gluster bricks ? Health of controller and disks ? - Is the FS aligned properly to the HW controller (stripe unit and stripe width )? More details on https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/ht... - FS Fragmentation ? Is your FS full ? - Gluster volume status ? Client count ? (for example: gluster volume status all client-list) - Gluster version, cluster.op-version and cluster.max-op-version ? - If your network is slower than the bricks' backend (disks faster than network) , you can set cluster.choose-local to 'yes' - Any errors and warnings in the gluster logs ? Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 13:59:04 Гринуич+3, <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU...

Well, each volume has only one brick. Each volume has one dedicated server. Each server storage has the disks in raid10. I made the gluster volume like this gluster volume create data transport tcp gfs.domain.com:/home/brick1 And than create a data domain in ovirt: gfs.domain.com:/data is this correct? Thanks José De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 17:09:26 Assunto: Re: [ovirt-users] Improve glusterfs performance Why don't you use the devices of the 2 bricks in a single striped LV or raid0 md device ? Distributed volumes spread the files among the bricks and the performance is limited to the brick's device speed. Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 18:09:47 Гринуич+2, <suporte@logicworks.pt> написа: Yes, the VM's disks are on the distributed volume They were created like this: gluster volume create data transport tcp gfs2.domain.com:/home/brick1 ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 15:53:57 Assunto: Re: [ovirt-users] Improve glusterfs performance Are the VM's disks located on the distributed Volume ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 17:21:49 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: The engine version is 4.3.9.4-1.el7 I have 2 simple glusterfs, a volume with one brick gfs1 with an old version of gluster (3.7.6) and a volume with one brick gfs2 with version 6.8 2 hosts, node3 is up to date, node 2 is not up to date: RHEL - 7 – 7.1908.0.el7.centos CentOS Linux 7 (Core) 3.10.0 – 1062.18.1.el7.x86_64 2.12.0 – 33.1.el7_7.4 libvirt-4.5.0-23.el7_7.6 vdsm-4.30.43-1.el7 0.14.0 – 7.el7 glusterfs-6.8-1.el7 librbd1-10.2.5-4.el7 openvswitch-2.11.0-4.el7 gfs1 has 3 nic's gigabit in mode 802.3ad gfs2 has 4 nic's gigabit in mode 802.3ad A VM performes better in node3/gfs1: # dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 registos dentro 1+0 registos fora 1073741824 bytes (1,1 GB) copiados, 12,3365 s, 87,0 MB/s in node2/gfs2: dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 51.0159 s, 21.0 MB/s For windows machine is the same. It helps if you have windows drives up to date. ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Domingo, 25 De Outubro de 2020 19:58:12 Assunto: Re: [ovirt-users] Improve glusterfs performance It seems that your e-mail went to the spam. I would start with isolating the issue ? 1. Is this a VM specific issue or a more-wide issue. - Can you move another VM on that storage domain and verify performance ? - Can you create/migrate same OS-type VM and check performance? - What about running a VM with different version of Windows or even better -> Linux ? Also I would check the systems from bottom up. Check the following: - Are you using Latest Firmware/OS updates for the Hosts and Engine ? - What is your cmdline : cat /proc/cmdline - Tuned profile - Are you using a HW controller for the Gluster bricks ? Health of controller and disks ? - Is the FS aligned properly to the HW controller (stripe unit and stripe width )? More details on https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/ht... - FS Fragmentation ? Is your FS full ? - Gluster volume status ? Client count ? (for example: gluster volume status all client-list) - Gluster version, cluster.op-version and cluster.max-op-version ? - If your network is slower than the bricks' backend (disks faster than network) , you can set cluster.choose-local to 'yes' - Any errors and warnings in the gluster logs ? Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 13:59:04 Гринуич+3, <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU...

No, is not a replica gluster, is just one brick one volume, one single server storage. This is what I get: # gluster volume set data group virt volume set: failed: Cannot set cluster.shd-max-threads for a non-replicate volume. Since I don't have a replica volume I cannot optimize it for oVirt? So, it's better to use an iScsi or NFS for a single storage server? Thanks José De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 19:20:21 Assunto: Re: [ovirt-users] Improve glusterfs performance I was left with the impression that you got 2 bricks (2 raids) on a single server. As I understand , you cannot get a 'replica 3 arbiter 1' or a 'replica 3' volumes , right ? When you create your volume , you need to use the UI's dropdown "Optimize for virt" or the command 'gluster volume set <VOLUME> group virt' in order to implement the optimal settings for Virtualization.Of course , some tunings can be made like how many I/O threads to be used and some caching - but even the defaults are OK. Have you checked anything from the list I send you ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 19:31:15 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: Well, each volume has only one brick. Each volume has one dedicated server. Each server storage has the disks in raid10. I made the gluster volume like this gluster volume create data transport tcp gfs.domain.com:/home/brick1 And than create a data domain in ovirt: gfs.domain.com:/data is this correct? Thanks José ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 17:09:26 Assunto: Re: [ovirt-users] Improve glusterfs performance Why don't you use the devices of the 2 bricks in a single striped LV or raid0 md device ? Distributed volumes spread the files among the bricks and the performance is limited to the brick's device speed. Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 18:09:47 Гринуич+2, <suporte@logicworks.pt> написа: Yes, the VM's disks are on the distributed volume They were created like this: gluster volume create data transport tcp gfs2.domain.com:/home/brick1 ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 15:53:57 Assunto: Re: [ovirt-users] Improve glusterfs performance Are the VM's disks located on the distributed Volume ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 17:21:49 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: The engine version is 4.3.9.4-1.el7 I have 2 simple glusterfs, a volume with one brick gfs1 with an old version of gluster (3.7.6) and a volume with one brick gfs2 with version 6.8 2 hosts, node3 is up to date, node 2 is not up to date: RHEL - 7 – 7.1908.0.el7.centos CentOS Linux 7 (Core) 3.10.0 – 1062.18.1.el7.x86_64 2.12.0 – 33.1.el7_7.4 libvirt-4.5.0-23.el7_7.6 vdsm-4.30.43-1.el7 0.14.0 – 7.el7 glusterfs-6.8-1.el7 librbd1-10.2.5-4.el7 openvswitch-2.11.0-4.el7 gfs1 has 3 nic's gigabit in mode 802.3ad gfs2 has 4 nic's gigabit in mode 802.3ad A VM performes better in node3/gfs1: # dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 registos dentro 1+0 registos fora 1073741824 bytes (1,1 GB) copiados, 12,3365 s, 87,0 MB/s in node2/gfs2: dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 51.0159 s, 21.0 MB/s For windows machine is the same. It helps if you have windows drives up to date. ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Domingo, 25 De Outubro de 2020 19:58:12 Assunto: Re: [ovirt-users] Improve glusterfs performance It seems that your e-mail went to the spam. I would start with isolating the issue ? 1. Is this a VM specific issue or a more-wide issue. - Can you move another VM on that storage domain and verify performance ? - Can you create/migrate same OS-type VM and check performance? - What about running a VM with different version of Windows or even better -> Linux ? Also I would check the systems from bottom up. Check the following: - Are you using Latest Firmware/OS updates for the Hosts and Engine ? - What is your cmdline : cat /proc/cmdline - Tuned profile - Are you using a HW controller for the Gluster bricks ? Health of controller and disks ? - Is the FS aligned properly to the HW controller (stripe unit and stripe width )? More details on https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/ht... - FS Fragmentation ? Is your FS full ? - Gluster volume status ? Client count ? (for example: gluster volume status all client-list) - Gluster version, cluster.op-version and cluster.max-op-version ? - If your network is slower than the bricks' backend (disks faster than network) , you can set cluster.choose-local to 'yes' - Any errors and warnings in the gluster logs ? Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 13:59:04 Гринуич+3, <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU...

On 10/28/20 12:46 PM, suporte@logicworks.pt wrote:

I think I have a problem in a Nic of one host. This host is the SPM That's probably why the gluster is slow! how can I change SPM to other host, by increasing SPM priority to high? On a different host in the cluster just select it to be the SPM, it should move.

You can change it via UI -> Hosts -> select new SPM host -> Management -> Select as SPM Best Regards, Strahil Nikolov В сряда, 28 октомври 2020 г., 19:46:14 Гринуич+2, <suporte@logicworks.pt> написа: I think I have a problem in a Nic of one host. This host is the SPM That's probably why the gluster is slow! how can I change SPM to other host, by increasing SPM priority to high? Thanks José ________________________________ De: suporte@logicworks.pt Para: "Strahil Nikolov" <hunter86_bg@yahoo.com> Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 19:52:53 Assunto: [ovirt-users] Re: Improve glusterfs performance No, is not a replica gluster, is just one brick one volume, one single server storage. This is what I get: # gluster volume set data group virt volume set: failed: Cannot set cluster.shd-max-threads for a non-replicate volume. Since I don't have a replica volume I cannot optimize it for oVirt? So, it's better to use an iScsi or NFS for a single storage server? Thanks José ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 19:20:21 Assunto: Re: [ovirt-users] Improve glusterfs performance I was left with the impression that you got 2 bricks (2 raids) on a single server. As I understand , you cannot get a 'replica 3 arbiter 1' or a 'replica 3' volumes , right ? When you create your volume , you need to use the UI's dropdown "Optimize for virt" or the command 'gluster volume set <VOLUME> group virt' in order to implement the optimal settings for Virtualization.Of course , some tunings can be made like how many I/O threads to be used and some caching - but even the defaults are OK. Have you checked anything from the list I send you ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 19:31:15 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: Well, each volume has only one brick. Each volume has one dedicated server. Each server storage has the disks in raid10. I made the gluster volume like this gluster volume create data transport tcp gfs.domain.com:/home/brick1 And than create a data domain in ovirt: gfs.domain.com:/data is this correct? Thanks José ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 17:09:26 Assunto: Re: [ovirt-users] Improve glusterfs performance Why don't you use the devices of the 2 bricks in a single striped LV or raid0 md device ? Distributed volumes spread the files among the bricks and the performance is limited to the brick's device speed. Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 18:09:47 Гринуич+2, <suporte@logicworks.pt> написа: Yes, the VM's disks are on the distributed volume They were created like this: gluster volume create data transport tcp gfs2.domain.com:/home/brick1 ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 15:53:57 Assunto: Re: [ovirt-users] Improve glusterfs performance Are the VM's disks located on the distributed Volume ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 17:21:49 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: The engine version is 4.3.9.4-1.el7 I have 2 simple glusterfs, a volume with one brick gfs1 with an old version of gluster (3.7.6) and a volume with one brick gfs2 with version 6.8 2 hosts, node3 is up to date, node 2 is not up to date: RHEL - 7 – 7.1908.0.el7.centos CentOS Linux 7 (Core) 3.10.0 – 1062.18.1.el7.x86_64 2.12.0 – 33.1.el7_7.4 libvirt-4.5.0-23.el7_7.6 vdsm-4.30.43-1.el7 0.14.0 – 7.el7 glusterfs-6.8-1.el7 librbd1-10.2.5-4.el7 openvswitch-2.11.0-4.el7 gfs1 has 3 nic's gigabit in mode 802.3ad gfs2 has 4 nic's gigabit in mode 802.3ad A VM performes better in node3/gfs1: # dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync  1+0 registos dentro 1+0 registos fora 1073741824 bytes (1,1 GB) copiados, 12,3365 s, 87,0 MB/s in node2/gfs2: dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync  1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 51.0159 s, 21.0 MB/s For windows machine is the same. It helps if you have windows drives up to date. ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Domingo, 25 De Outubro de 2020 19:58:12 Assunto: Re: [ovirt-users] Improve glusterfs performance It seems that your e-mail went to the spam. I would start with isolating the issue ? 1. Is this a VM specific issue or a more-wide issue. - Can you move another VM on that storage domain and verify performance ? - Can you create/migrate same OS-type VM and check performance? - What about running a VM with different version of Windows or even better -> Linux ? Also I would check the systems from bottom up. Check the following: - Are you using Latest Firmware/OS updates for the Hosts and Engine ? - What is your cmdline : cat /proc/cmdline - Tuned profile - Are you using a HW controller for the Gluster bricks ? Health of controller and disks ? - Is the FS aligned properly to the HW controller (stripe unit and stripe width )? More details on https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/ht... - FS Fragmentation ? Is your FS full ? - Gluster volume status ? Client count ? (for example: gluster volume status all client-list) - Gluster version, cluster.op-version and cluster.max-op-version ? - If your network is slower than the bricks' backend (disks faster than network) , you can set cluster.choose-local to 'yes' - Any errors and warnings in the gluster logs ? Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 13:59:04 Гринуич+3, <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU... _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NYM3OOCPUVFNRK...

The actual command is : gluster volume set help | less optimize for virt is just applying some settings optimal for virtualization tasks - you can do it on any type of volume It is up to you , the good stuff with Gluster is that you can easily scale it out and this is not so easy with NFS/iSCSI Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 21:53:05 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: No, is not a replica gluster, is just one brick one volume, one single server storage. This is what I get: # gluster volume set data group virt volume set: failed: Cannot set cluster.shd-max-threads for a non-replicate volume. Since I don't have a replica volume I cannot optimize it for oVirt? So, it's better to use an iScsi or NFS for a single storage server? Thanks José ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 19:20:21 Assunto: Re: [ovirt-users] Improve glusterfs performance I was left with the impression that you got 2 bricks (2 raids) on a single server. As I understand , you cannot get a 'replica 3 arbiter 1' or a 'replica 3' volumes , right ? When you create your volume , you need to use the UI's dropdown "Optimize for virt" or the command 'gluster volume set <VOLUME> group virt' in order to implement the optimal settings for Virtualization.Of course , some tunings can be made like how many I/O threads to be used and some caching - but even the defaults are OK. Have you checked anything from the list I send you ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 19:31:15 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: Well, each volume has only one brick. Each volume has one dedicated server. Each server storage has the disks in raid10. I made the gluster volume like this gluster volume create data transport tcp gfs.domain.com:/home/brick1 And than create a data domain in ovirt: gfs.domain.com:/data is this correct? Thanks José ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 17:09:26 Assunto: Re: [ovirt-users] Improve glusterfs performance Why don't you use the devices of the 2 bricks in a single striped LV or raid0 md device ? Distributed volumes spread the files among the bricks and the performance is limited to the brick's device speed. Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 18:09:47 Гринуич+2, <suporte@logicworks.pt> написа: Yes, the VM's disks are on the distributed volume They were created like this: gluster volume create data transport tcp gfs2.domain.com:/home/brick1 ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Terça-feira, 27 De Outubro de 2020 15:53:57 Assunto: Re: [ovirt-users] Improve glusterfs performance Are the VM's disks located on the distributed Volume ? Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 17:21:49 Гринуич+2, suporte@logicworks.pt <suporte@logicworks.pt> написа: The engine version is 4.3.9.4-1.el7 I have 2 simple glusterfs, a volume with one brick gfs1 with an old version of gluster (3.7.6) and a volume with one brick gfs2 with version 6.8 2 hosts, node3 is up to date, node 2 is not up to date: RHEL - 7 – 7.1908.0.el7.centos CentOS Linux 7 (Core) 3.10.0 – 1062.18.1.el7.x86_64 2.12.0 – 33.1.el7_7.4 libvirt-4.5.0-23.el7_7.6 vdsm-4.30.43-1.el7 0.14.0 – 7.el7 glusterfs-6.8-1.el7 librbd1-10.2.5-4.el7 openvswitch-2.11.0-4.el7 gfs1 has 3 nic's gigabit in mode 802.3ad gfs2 has 4 nic's gigabit in mode 802.3ad A VM performes better in node3/gfs1: # dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync  1+0 registos dentro 1+0 registos fora 1073741824 bytes (1,1 GB) copiados, 12,3365 s, 87,0 MB/s in node2/gfs2: dd if=/dev/zero of=test1.img bs=1G count=1 oflag=dsync  1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 51.0159 s, 21.0 MB/s For windows machine is the same. It helps if you have windows drives up to date. ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: suporte@logicworks.pt Cc: "users" <users@ovirt.org> Enviadas: Domingo, 25 De Outubro de 2020 19:58:12 Assunto: Re: [ovirt-users] Improve glusterfs performance It seems that your e-mail went to the spam. I would start with isolating the issue ? 1. Is this a VM specific issue or a more-wide issue. - Can you move another VM on that storage domain and verify performance ? - Can you create/migrate same OS-type VM and check performance? - What about running a VM with different version of Windows or even better -> Linux ? Also I would check the systems from bottom up. Check the following: - Are you using Latest Firmware/OS updates for the Hosts and Engine ? - What is your cmdline : cat /proc/cmdline - Tuned profile - Are you using a HW controller for the Gluster bricks ? Health of controller and disks ? - Is the FS aligned properly to the HW controller (stripe unit and stripe width )? More details on https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/ht... - FS Fragmentation ? Is your FS full ? - Gluster volume status ? Client count ? (for example: gluster volume status all client-list) - Gluster version, cluster.op-version and cluster.max-op-version ? - If your network is slower than the bricks' backend (disks faster than network) , you can set cluster.choose-local to 'yes' - Any errors and warnings in the gluster logs ? Best Regards, Strahil Nikolov В четвъртък, 22 октомври 2020 г., 13:59:04 Гринуич+3, <suporte@logicworks.pt> написа: Hello, For example, a window machine runs to slow, usually the disk is allways in 100% The group virt settings is this?: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.low-prio-threads=32 network.remote-dio=enable features.shard=on user.cifs=off client.event-threads=4 server.event-threads=4 performance.client-io-threads=on ________________________________ De: "Strahil Nikolov" <hunter86_bg@yahoo.com> Para: "users" <users@ovirt.org>, suporte@logicworks.pt Enviadas: Quarta-feira, 21 De Outubro de 2020 19:22:14 Assunto: Re: [ovirt-users] Improve glusterfs performance Usually, oVirt uses the 'virt' group of settings. What are you symptoms ? Best Regards, Strahil Nikolov В сряда, 21 октомври 2020 г., 16:44:50 Гринуич+3, suporte@logicworks.pt <suporte@logicworks.pt> написа: Hello, Can anyone help me in how can I improve the performance of glusterfs to work with oVirt? Thanks -- ________________________________ Jose Ferradeira http://www.logicworks.pt _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KPU4QFTMUMMFGU...