[ovirt-users] gluster performance oVirt 3.4

[Sahina Bose]

On 05/14/2014 02:55 PM, Vadims Korsaks wrote:
> Citējot "Sahina Bose" <sabose at redhat.com>:
>> On 05/14/2014 02:36 PM, Vadims Korsaks wrote:
>>> Citējot Sahina Bose <sabose at redhat.com> :
>>>> On 05/13/2014 07:27 PM, Vadims Korsaks wrote:
>>>>> Citējot Humble Devassy Chirammal
>>>>> <humble.devassy at gmail.com> :
>>>>>>> Citējot Vijay Bellur <vbellur at redhat.com> :
>>>>>>>> On 05/11/2014 02:04 AM, Vadims Korsaks
>>> wrote:
>>>>>>>>> HI!
>>>>>>>>>
>>>>>>>>> Created 2 node setup with oVirt 3.4 and
>>>>>>> CentOS 6.5, for storage created
>>>>>>>>> 2 node replicated gluster (3.5) fs on same
>>>>>>> hosts with oVirt.
>>>>>>>>> mount looks like this:
>>>>>>>>> 127.0.0.1:/gluster01 on
>>>>>>>>>
> /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01
>>>>>>> type fuse.glusterfs
> (rw,default_permissions,allow_other,max_read=131072)
>>>>>>>>> when i making gluster test with dd,
>>>>> something
>>>>>>> like
>>>>>>>>> dd if=/dev/zero bs=1M count=20000
>>>>>>>>>
> of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka
>>>>>>>>> i'm gettting speed ~ 110 MB/s, so this is
>>>>>>> 1Gbps speed of ethernet adapter
>>>>>>>>> but with in VM created in oVirt speed is
>>>>>>> lower than 20 MB/s
>>>>>>>>> why there is so huge difference?
>>>>>>>>> how can improve VMs disks speed?
>>>>>>>>>
>>>>>>>> What are your gluster volume settings?
>>>>> Have you
>>>>>>> applied the following
>>>>>>>> performance tunables in gluster's virt
>>>>> profile:
>>>>>>>> eager-lock=enable
>>>>>>>> remote-dio=enable
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Vijay
>>>>>>>>
>>>>>>> setting were:
>>>>>>> [root at centos155 ~]# gluster volume info
>>> gluster01
>>>>>>> Volume Name: gluster01
>>>>>>> Type: Replicate
>>>>>>> Volume ID:
>>> 436edaa3-ac8b-421f-aa35-68b5bd7064b6
>>>>>>> Status: Started
>>>>>>> Number of Bricks: 1 x 2 = 2
>>>>>>> Transport-type: tcp
>>>>>>> Bricks:
>>>>>>> Brick1: 10.2.75.152:/mnt/gluster01/brick
>>>>>>> Brick2: 10.2.75.155:/mnt/gluster01/brick
>>>>>>> Options Reconfigured:
>>>>>>> storage.owner-gid: 36
>>>>>>> storage.owner-uid: 36
>>>>>>>
>>>>>>>
>>>>>>> add your settings settings now it looks
>>>>>>>
>>>>>>> [root at centos155 ~]# gluster volume info
>>> gluster01
>>>>>>> Volume Name: gluster01
>>>>>>> Type: Replicate
>>>>>>> Volume ID:
>>> 436edaa3-ac8b-421f-aa35-68b5bd7064b6
>>>>>>> Status: Started
>>>>>>> Number of Bricks: 1 x 2 = 2
>>>>>>> Transport-type: tcp
>>>>>>> Bricks:
>>>>>>> Brick1: 10.2.75.152:/mnt/gluster01/brick
>>>>>>> Brick2: 10.2.75.155:/mnt/gluster01/brick
>>>>>>> Options Reconfigured:
>>>>>>> network.remote-dio: enable
>>>>>>> cluster.eager-lock: enable
>>>>>>> storage.owner-gid: 36
>>>>>>> storage.owner-uid: 36
>>>>>>>
>>>>>>>
>>>>>>> but this didn't affected performace in any
>>>>> big way
>>>>>>> should hosts to be restarted?
>>>>>>>
>>>>>> glusterfs storage domain configuration GUI got
>>>>> ""Optimize for virt. store" option which have to
>>>>> be enabled when configuring for virt store.
>>>>>>
> Ref#http://www.ovirt.org/Features/GlusterFS_Storage_Domain
>>>>>> If the configuration is manual, you need to
>>>>> set the group to 'virt' as shown below
>>>>>> #gluster volume set VOLNAME group virt
>>>>>>
>>>>>> This will enable below options on gluster
>>> volume :
>>>>>> quick-read=off
>>>>>> read-ahead=off
>>>>>> io-cache=off
>>>>>> stat-prefetch=off
>>>>>> eager-lock=enable
>>>>>> remote-dio=on
>>>>>>
>>>>>>
>>>>>> Can you please make sure group has set
>>> properly ?
>>>>>> Also, invoke "dd" with oflag=direct option and
>>>>> check whether it helps ..
>>>>>>
>>>>>> --Humble
>>>>> tnx a lot!! now it's much better - from VM i can
>>>>> get dd with ~ 60 MB/s
>>>>> this is still ~ x2 lower than from host, but x3
>>>>> better than it was before :)
>>>>>
>>>>> BTW could not found "GUI got ""Optimize for virt.
>>>>> store" option" in oVirt 3.5
>>>>
>>>> The option "Optimize for Virt store" is
>>> available when you select a
>>>> volume in Ovirt - both as a right click menu
>>> option as well as a button
>>>> in the top sub navigation bar.
>>>>
>>>> You also can check this option while creating a
>>> gluster volume using the
>>>> oVirt GUI
>>>>
>>>>
>>> i have glusterfs as my master storage, but there
>>> is nothing in volumes, empty. And i can't create
>>> gluster volume, there is no choise in Data Center
>>> and Volume Cluster.
>>>
>>
>> To be able to manage gluster (i.e create volume,
> add bricks and peer
>> probe hosts) from Ovirt UI, the cluster should
> have the "Enable gluster
>> service" option checked. I think you may not
> have enabled this when you
>> created the cluster. To use gluster volume as
> storage domain, this
>> option is not required.
>>
>> Hope this helps.
>>
>>
> Ok, i see, tnx a lot.
> any chances to change this for Default Cluster,
> because i have self hosted engine and just 2 hosts?



You can edit the cluster and change this option. Please make sure that 
vdsm-gluster is installed on the hosts in the cluster and a gluster peer 
probe has been done if there are more than one hosts in the cluster.


>