Re: [ovirt-users] gluster performance oVirt 3.4
Citējot "Sahina Bose" <sabose@redhat.com>:
On 05/14/2014 02:36 PM, Vadims Korsaks wrote:
Citējot Sahina Bose <sabose@redhat.com> :
On 05/13/2014 07:27 PM, Vadims Korsaks wrote:
Citējot Humble Devassy Chirammal <humble.devassy@gmail.com> :
Citējot Vijay Bellur <vbellur@redhat.com> : > On 05/11/2014 02:04 AM, Vadims Korsaks
wrote:
>> HI! >> >> Created 2 node setup with oVirt 3.4 and CentOS 6.5, for storage created >> 2 node replicated gluster (3.5) fs on same hosts with oVirt. >> mount looks like this: >> 127.0.0.1:/gluster01 on >>
/rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01
type fuse.glusterfs >>
(rw,default_permissions,allow_other,max_read=131072)
>> >> when i making gluster test with dd, something like >> dd if=/dev/zero bs=1M count=20000 >>
of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka
>> i'm gettting speed ~ 110 MB/s, so this is 1Gbps speed of ethernet adapter >> >> but with in VM created in oVirt speed is lower than 20 MB/s >> >> why there is so huge difference? >> how can improve VMs disks speed? >> > > What are your gluster volume settings? Have you applied the following > performance tunables in gluster's virt profile: > > eager-lock=enable > remote-dio=enable > > Regards, > Vijay > setting were: [root@centos155 ~]# gluster volume info gluster01
Volume Name: gluster01 Type: Replicate Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.2.75.152:/mnt/gluster01/brick Brick2: 10.2.75.155:/mnt/gluster01/brick Options Reconfigured: storage.owner-gid: 36 storage.owner-uid: 36
add your settings settings now it looks
[root@centos155 ~]# gluster volume info gluster01
Volume Name: gluster01 Type: Replicate Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.2.75.152:/mnt/gluster01/brick Brick2: 10.2.75.155:/mnt/gluster01/brick Options Reconfigured: network.remote-dio: enable cluster.eager-lock: enable storage.owner-gid: 36 storage.owner-uid: 36
but this didn't affected performace in any big way should hosts to be restarted?
glusterfs storage domain configuration GUI got ""Optimize for virt. store" option which have to be enabled when configuring for virt store.
Ref#http://www.ovirt.org/Features/GlusterFS_Storage_Domain
If the configuration is manual, you need to
set the group to 'virt' as shown below
#gluster volume set VOLNAME group virt
This will enable below options on gluster
volume :
quick-read=off read-ahead=off io-cache=off stat-prefetch=off eager-lock=enable remote-dio=on
Can you please make sure group has set
properly ?
Also, invoke "dd" with oflag=direct option and
check whether it helps ..
--Humble
tnx a lot!! now it's much better - from VM i can get dd with ~ 60 MB/s this is still ~ x2 lower than from host, but x3 better than it was before :)
BTW could not found "GUI got ""Optimize for virt. store" option" in oVirt 3.5
The option "Optimize for Virt store" is available when you select a volume in Ovirt - both as a right click menu option as well as a button in the top sub navigation bar.
You also can check this option while creating a gluster volume using the oVirt GUI
i have glusterfs as my master storage, but there is nothing in volumes, empty. And i can't create gluster volume, there is no choise in Data Center and Volume Cluster.
To be able to manage gluster (i.e create volume, add bricks and peer probe hosts) from Ovirt UI, the cluster should have the "Enable gluster service" option checked. I think you may not have enabled this when you created the cluster. To use gluster volume as storage domain, this option is not required.
Hope this helps.
Ok, i see, tnx a lot. any chances to change this for Default Cluster, because i have self hosted engine and just 2 hosts?
On 05/14/2014 02:55 PM, Vadims Korsaks wrote:
Citējot "Sahina Bose" <sabose@redhat.com>:
On 05/14/2014 02:36 PM, Vadims Korsaks wrote:
On 05/13/2014 07:27 PM, Vadims Korsaks wrote:
> Citējot Vijay Bellur <vbellur@redhat.com> : >> On 05/11/2014 02:04 AM, Vadims Korsaks wrote: >>> HI! >>> >>> Created 2 node setup with oVirt 3.4 and > CentOS 6.5, for storage created >>> 2 node replicated gluster (3.5) fs on same > hosts with oVirt. >>> mount looks like this: >>> 127.0.0.1:/gluster01 on >>> /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 > type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) >>> when i making gluster test with dd, something > like >>> dd if=/dev/zero bs=1M count=20000 >>> of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka >>> i'm gettting speed ~ 110 MB/s, so this is > 1Gbps speed of ethernet adapter >>> but with in VM created in oVirt speed is > lower than 20 MB/s >>> why there is so huge difference? >>> how can improve VMs disks speed? >>> >> What are your gluster volume settings? Have you > applied the following >> performance tunables in gluster's virt
Citējot Humble Devassy Chirammal <humble.devassy@gmail.com> : profile:
>> eager-lock=enable >> remote-dio=enable >> >> Regards, >> Vijay >> > setting were: > [root@centos155 ~]# gluster volume info gluster01 > Volume Name: gluster01 > Type: Replicate > Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 > Status: Started > Number of Bricks: 1 x 2 = 2 > Transport-type: tcp > Bricks: > Brick1: 10.2.75.152:/mnt/gluster01/brick > Brick2: 10.2.75.155:/mnt/gluster01/brick > Options Reconfigured: > storage.owner-gid: 36 > storage.owner-uid: 36 > > > add your settings settings now it looks > > [root@centos155 ~]# gluster volume info gluster01 > Volume Name: gluster01 > Type: Replicate > Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 > Status: Started > Number of Bricks: 1 x 2 = 2 > Transport-type: tcp > Bricks: > Brick1: 10.2.75.152:/mnt/gluster01/brick > Brick2: 10.2.75.155:/mnt/gluster01/brick > Options Reconfigured: > network.remote-dio: enable > cluster.eager-lock: enable > storage.owner-gid: 36 > storage.owner-uid: 36 > > > but this didn't affected performace in any big way > should hosts to be restarted? > glusterfs storage domain configuration GUI got ""Optimize for virt. store" option which have to be enabled when configuring for virt store.
Ref#http://www.ovirt.org/Features/GlusterFS_Storage_Domain
If the configuration is manual, you need to set the group to 'virt' as shown below #gluster volume set VOLNAME group virt
This will enable below options on gluster volume : quick-read=off read-ahead=off io-cache=off stat-prefetch=off eager-lock=enable remote-dio=on
Can you please make sure group has set
Also, invoke "dd" with oflag=direct option and check whether it helps ..
--Humble tnx a lot!! now it's much better - from VM i can get dd with ~ 60 MB/s this is still ~ x2 lower than from host, but x3 better than it was before :)
BTW could not found "GUI got ""Optimize for virt. store" option" in oVirt 3.5
The option "Optimize for Virt store" is available when you select a volume in Ovirt - both as a right click menu
Citējot Sahina Bose <sabose@redhat.com> : properly ? option as well as a button
in the top sub navigation bar.
You also can check this option while creating a gluster volume using the oVirt GUI
i have glusterfs as my master storage, but there is nothing in volumes, empty. And i can't create gluster volume, there is no choise in Data Center and Volume Cluster.
To be able to manage gluster (i.e create volume, add bricks and peer probe hosts) from Ovirt UI, the cluster should have the "Enable gluster service" option checked. I think you may not have enabled this when you created the cluster. To use gluster volume as storage domain, this option is not required.
Hope this helps.
Ok, i see, tnx a lot. any chances to change this for Default Cluster, because i have self hosted engine and just 2 hosts?
You can edit the cluster and change this option. Please make sure that vdsm-gluster is installed on the hosts in the cluster and a gluster peer probe has been done if there are more than one hosts in the cluster.
participants (2)
-
Sahina Bose -
Vadims Korsaks