12:59 p.m.
--PW0Eas8rCkcu1VkF
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Apr 12, 2016 at 11:11:54AM +0200, Roderick Mooi wrote:
Hi
=20
> It is not removed. Can you try 'gluster volume set volname cluster.eage=
r-lock enable`?
=20
This works. BTW by default this setting is =E2=80=9Con=E2=80=9D
Thanks for reporting back!
What=E2=80=99s the difference between =E2=80=9Con=E2=80=9D and
=E2=80=9Ce=
nable=E2=80=9D?
Both are the same, you could also use "yes", "true" and possibly
others.
Cheers,
Niels
=20
Thanks for the clarification.
=20
Regards,
=20
Roderick
=20
> On 06 Apr 2016, at 10:56 AM, Ravishankar N <ravishankar(a)redhat.com> wro=
te:
>=20
> On 04/06/2016 02:08 PM, Roderick Mooi wrote:
>> Hi Ravi and colleagues
>>=20
>> (apologies for hijacking this thread but I=E2=80=99m not sure where el=
se
to report this (and it is related).)
>>=20
>> With gluster 3.7.10, running
>> #gluster volume set <volname> group virt
>> fails with:
>> volume set: failed: option : eager-lock does not exist
>> Did you mean eager-lock?
>>=20
>> I had to remove the eager-lock setting from /var/lib/glusterd/groups/v=
irt to get this to work. It seems like setting eager-lock has been removed =
=66rom latest gluster. Is this correct? Either way, is there anything else =
I should do?
>=20
> It is not removed. Can you try 'gluster volume set volname cluster.eage=
r-lock enable`?
> I think the disperse (EC) translator introduced a
`disperse.eager-lock`=
which is why you would need to mention entire volume option
name to avoid =
ambiguity.
> We probably need to fix the virt profile setting to include the
entire =
name. By the way 'gluster volume set help` should give you the list of
all =
options.
>=20
> -Ravi
>=20
>>=20
>> Cheers,
>>=20
>> Roderick
>>=20
>>> On 12 Feb 2016, at 6:18 AM, Ravishankar N <ravishankar(a)redhat.com <ma=
ilto:ravishankar@redhat.com>> wrote:
>>>=20
>>> Hi Bill,
>>> Can you enable virt-profile setting for your volume and see if that h=
elps? You need to enable this optimization when you create the volume using=
ovrit, or use the following command for an existing volume:
>>>=20
>>> #gluster volume set <volname> group virt
>>>=20
>>> -Ravi
>>>=20
>>>=20
>>> On 02/12/2016 05:22 AM, Bill James wrote:
>>>> My apologies, I'm showing how much of a noob I am.
>>>> Ignore last direct to gluster numbers, as that wasn't really
gluster=
fs.
>>>>=20
>>>>=20
>>>> [root@ovirt2 test ~]# mount -t glusterfs ovirt2-ks.test.j2noc.com <h=
ttp://ovirt2-ks.test.j2noc.com/>:/gv1 /mnt/tmp/
>>>> [root@ovirt2 test ~]# time dd if=3D/dev/zero
of=3D/mnt/tmp/testfile2=
bs=3D1M count=3D1000 oflag=3Ddirect
>>>> 1048576000 bytes (1.0 GB) copied, 65.8596 s, 15.9
MB/s
>>>>=20
>>>> That's more how I expected, it is pointing to glusterfs
performance.
>>>>=20
>>>>=20
>>>>=20
>>>> On 02/11/2016 03:27 PM, Bill James wrote:
>>>>> don't know if it helps, but I ran a few more tests, all from the
sa=
me hardware node.
>>>>>=20
>>>>> The VM:
>>>>> [root@billjov1 ~]# time dd if=3D/dev/zero of=3D/root/testfile bs=3D=
1M count=3D1000 oflag=3Ddirect
>>>>> 1048576000 bytes (1.0 GB) copied, 62.5535 s,
16.8 MB/s
>>>>>=20
>>>>> Writing directly to gluster volume:
>>>>> [root@ovirt2 test ~]# time dd if=3D/dev/zero of=3D/gluster-store/br=
ick1/gv1/testfile bs=3D1M count=3D1000 oflag=3Ddirect
>>>>> 1048576000 bytes (1.0 GB) copied, 9.92048 s, 106
MB/s
>>>>>=20
>>>>>=20
>>>>> Writing to NFS volume:
>>>>> [root@ovirt2 test ~]# time dd if=3D/dev/zero of=3D/mnt/storage/qa/t=
estfile bs=3D1M count=3D1000 oflag=3Ddirect
>>>>> 1048576000 bytes (1.0 GB) copied, 10.5776 s,
99.1 MB/s
>>>>>=20
>>>>> NFS & Gluster are using the same interface. Tests were not run
at s=
ame time.
>>>>>=20
>>>>> This would suggest my problem isn't glusterfs, but the VM
performan=
ce.
>>>>>=20
>>>>>=20
>>>>>=20
>>>>> On 02/11/2016 03:13 PM, Bill James wrote:
>>>>>> xml attached.=20
>>>>>>=20
>>>>>>=20
>>>>>> On 02/11/2016 12:28 PM, Nir Soffer wrote:=20
>>>>>>> On Thu, Feb 11, 2016 at 8:27 PM, Bill James
<mailto:bill.james@j=
2.com><bill.james(a)j2.com>
<mailto:bill.james@j2.com> wrote:=20
>>>>>>>> thank you for the reply.=20
>>>>>>>>=20
>>>>>>>> We setup gluster using the names associated with NIC 2
IP.=20
>>>>>>>> Brick1: ovirt1-ks.test.j2noc.com
<http://ovirt1-ks.test.j2noc.=
com/>:/gluster-store/brick1/gv1=20
>>>>>>>> Brick2: ovirt2-ks.test.j2noc.com
<http://ovirt2-ks.test.j2noc.=
com/>:/gluster-store/brick1/gv1=20
>>>>>>>> Brick3: ovirt3-ks.test.j2noc.com
<http://ovirt3-ks.test.j2noc.=
com/>:/gluster-store/brick1/gv1=20
>>>>>>>>=20
>>>>>>>> That's NIC 2's IP.=20
>>>>>>>> Using 'iftop -i eno2 -L 5 -t' :=20
>>>>>>>>=20
>>>>>>>> dd if=3D/dev/zero of=3D/root/testfile bs=3D1M
count=3D1000 oflag=
=3Ddirect=20
>>>>>>>> 1048576000 bytes (1.0 GB) copied,
68.0714 s, 15.4 MB/s=20
>>>>>>> Can you share the xml of this vm? You can find it in vdsm
log,=20
>>>>>>> at the time you start the vm.=20
>>>>>>>=20
>>>>>>> Or you can do (on the host):=20
>>>>>>>=20
>>>>>>> # virsh=20
>>>>>>> virsh # list=20
>>>>>>> (username: vdsm@ovirt password: shibboleth)=20
>>>>>>> virsh # dumpxml vm-id=20
>>>>>>>=20
>>>>>>>> Peak rate (sent/received/total):
281Mb 5.36=
Mb=20
>>>>>>>> 282Mb=20
>>>>>>>> Cumulative (sent/received/total):
1.96GB 14.6=
MB=20
>>>>>>>> 1.97GB=20
>>>>>>>>=20
>>>>>>>> gluster volume info gv1:=20
>>>>>>>> Options Reconfigured:=20
>>>>>>>> performance.write-behind-window-size: 4MB=20
>>>>>>>> performance.readdir-ahead: on=20
>>>>>>>> performance.cache-size: 1GB=20
>>>>>>>> performance.write-behind: off=20
>>>>>>>>=20
>>>>>>>> performance.write-behind: off didn't help.=20
>>>>>>>> Neither did any other changes I've tried.=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> There is no VM traffic on this VM right now except my
test.=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> On 02/10/2016 11:55 PM, Nir Soffer wrote:=20
>>>>>>>>> On Thu, Feb 11, 2016 at 2:42 AM, Ravishankar N
<mailto:ravisha=
nkar@redhat.com><ravishankar(a)redhat.com>
<mailto:ravishankar@redhat.com>=20
>>>>>>>>> wrote:=20
>>>>>>>>>> +gluster-users=20
>>>>>>>>>>=20
>>>>>>>>>> Does disabling
'performance.write-behind' give a better throug=
hput?=20
>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>> On 02/10/2016 11:06 PM, Bill James wrote:=20
>>>>>>>>>>> I'm setting up a ovirt cluster using
glusterfs and noticing n=
ot stellar=20
>>>>>>>>>>> performance.=20
>>>>>>>>>>> Maybe my setup could use some
adjustments?=20
>>>>>>>>>>>=20
>>>>>>>>>>> 3 hardware nodes running centos7.2,
glusterfs 3.7.6.1, ovirt =
3.6.2.6-1.=20
>>>>>>>>>>> Each node has 8 spindles
configured in 1 array which is split=
using LVM=20
>>>>>>>>>>> with one logical volume
for system and one for gluster.=20
>>>>>>>>>>> They each have 4 NICs,=20
>>>>>>>>>>> NIC1 =3D ovirtmgmt=20
>>>>>>>>>>> NIC2 =3D gluster (1GbE)=20
>>>>>>>>> How do you ensure that gluster trafic is using this
nic?=20
>>>>>>>>>=20
>>>>>>>>>>> NIC3 =3D VM traffic=20
>>>>>>>>> How do you ensure that vm trafic is using this
nic?=20
>>>>>>>>>=20
>>>>>>>>>>> I tried with default glusterfs settings=20
>>>>>>>>> And did you find any difference?=20
>>>>>>>>>=20
>>>>>>>>>>> and also with:=20
>>>>>>>>>>> performance.cache-size: 1GB=20
>>>>>>>>>>> performance.readdir-ahead: on=20
>>>>>>>>>>> performance.write-behind-window-size:
4MB=20
>>>>>>>>>>>=20
>>>>>>>>>>> [root@ovirt3 test scripts]# gluster volume
info gv1=20
>>>>>>>>>>>=20
>>>>>>>>>>> Volume Name: gv1=20
>>>>>>>>>>> Type: Replicate=20
>>>>>>>>>>> Volume ID:
71afc35b-09d7-4384-ab22-57d032a0f1a2=20
>>>>>>>>>>> Status: Started=20
>>>>>>>>>>> Number of Bricks: 1 x 3 =3D 3=20
>>>>>>>>>>> Transport-type: tcp=20
>>>>>>>>>>> Bricks:=20
>>>>>>>>>>> Brick1: ovirt1-ks.test.j2noc.com
<http://ovirt1-ks.test.j2noc=
=2Ecom/>:/gluster-store/brick1/gv1=20
>>>>>>>>>>> Brick2:
ovirt2-ks.test.j2noc.com <http://ovirt2-ks.test.j2noc=
=2Ecom/>:/gluster-store/brick1/gv1=20
>>>>>>>>>>> Brick3:
ovirt3-ks.test.j2noc.com <http://ovirt3-ks.test.j2noc=
=2Ecom/>:/gluster-store/brick1/gv1=20
>>>>>>>>>>> Options
Reconfigured:=20
>>>>>>>>>>> performance.cache-size: 1GB=20
>>>>>>>>>>> performance.readdir-ahead: on=20
>>>>>>>>>>> performance.write-behind-window-size:
4MB=20
>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>>>>>> Using simple dd test on VM in ovirt:=20
>>>>>>>>>>> dd if=3D/dev/zero of=3D/root/testfile
bs=3D1G count=3D1 o=
flag=3Ddirect=20
>>>>>>>>> block size of 1G?!=20
>>>>>>>>>=20
>>>>>>>>> Try 1M (our default for storage operations)=20
>>>>>>>>>=20
>>>>>>>>>>> 1073741824 bytes (1.1 GB) copied,
65.9337 s, 16.3 MB/=
s=20
>>>>>>>>>>>=20
>>>>>>>>>>> Another VM not in ovirt using nfs:=20
>>>>>>>>>>> dd if=3D/dev/zero of=3D/root/testfile
bs=3D1G count=3D1 =
oflag=3Ddirect=20
>>>>>>>>>>> 1073741824 bytes
(1.1 GB) copied, 27.0079 s, 39.8 MB/=
s=20
>>>>>>>>>>>=20
>>>>>>>>>>>=20
>>>>>>>>>>> Is that expected or is there a better way to
set it up to get=
better=20
>>>>>>>>>>> performance?=20
>>>>>>>>> Adding Niels for advice.=20
>>>>>>>>>=20
>>>>>>>>>>> This email, its contents and ....=20
>>>>>>>>> Please avoid this, this is a public mailing list,
everything yo=
u write=20
>>>>>>>>> here is public.=20
>>>>>>>>>=20
>>>>>>>>> Nir=20
>>>>>>>> I'll have to look into how to remove this sig for
this mailing l=
ist....=20
>>>>>>>>=20
>>>>>>>> Cloud Services for Business
<http://www.j2.com/>www.j2.com <htt=
p://www.j2.com/>=20
>>>>>>>> j2 | eFax | eVoice | FuseMail |
Campaigner | KeepItSafe | Onebox=
=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> This email, its contents and attachments contain
information fro=
m j2 Global,=20
>>>>>>>> Inc. and/or its affiliates which may
be privileged, confidential=
or=20
>>>>>>>> otherwise protected from disclosure.
The information is intended=
to be for=20
>>>>>>>> the addressee(s) only. If you are
not an addressee, any disclosu=
re, copy,=20
>>>>>>>> distribution, or use of the contents
of this message is prohibit=
ed. If you=20
>>>>>>>> have received this email in error
please notify the sender by re=
ply e-mail=20
>>>>>>>> and delete the original message and
any copies. (c) 2015 j2 Glob=
al, Inc. All=20
>>>>>>>> rights reserved. eFax, eVoice,
Campaigner, FuseMail, KeepItSafe,=
and Onebox=20
>>>>>>>> are registered trademarks of j2
Global, Inc. and its affiliates.=
=20
>>>>>>=20
>>>>>>=20
>>>>>>=20
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.=
org/mailman/listinfo/users>
ral&utm_campaign=3Demployeeemail>
>>>>> This email, its contents and attachments contain
information from j=
2 Global, Inc
<http://www.j2.com/?utm_source=3Dj2global&utm_medium=3Dxsell-=
referral&utm_campaign=3Demployemail>. and/or its affiliates which may be pr=
ivileged, confidential or otherwise protected from disclosure. The informat=
ion is intended to be for the addressee(s) only. If you are not an addresse=
e, any disclosure, copy, distribution, or use of the contents of this messa=
ge is prohibited. If you have received this email in error please notify th=
e sender by reply e-mail and delete the original message and any copies. =
=C2=A9 2015 j2 Global, Inc <http://www.j2.com/>;. All rights reserved. eFax =
=C2=AE <http://www.efax.com/>;, eVoice =C2=AE <http://www.evoice.com/>;, Camp=
aigner =C2=AE <http://www.campaigner.com/>;, FuseMail =C2=AE <http://www.fus=
email.com/>;, KeepItSafe =C2=AE <http://www.keepitsafe.com/> and Onebox =C2=
=AE <http://www.onebox.com/> are r egistered trademarks of j2 Global, Inc <=
http://www.j2.com/>;. and its affiliates.
>>>>>=20
>>>>>=20
>>>>>=20
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>>>> http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.o=
rg/mailman/listinfo/users>
al&utm_campaign=3Demployeeemail>
>>>> This email, its contents and attachments contain
information from j2=
Global, Inc
<http://www.j2.com/?utm_source=3Dj2global&utm_medium=3Dxsell-r=
eferral&utm_campaign=3Demployemail>. and/or its affiliates which may be pri=
vileged, confidential or otherwise protected from disclosure. The informati=
on is intended to be for the addressee(s) only. If you are not an addressee=
, any disclosure, copy, distribution, or use of the contents of this messag=
e is prohibited. If you have received this email in error please notify the=
sender by reply e-mail and delete the original message and any copies. =C2=
=A9 2015 j2 Global, Inc <http://www.j2.com/>;. All rights reserved. eFax =C2=
=AE <http://www.efax.com/>;, eVoice =C2=AE <http://www.evoice.com/>;, Campaig=
ner =C2=AE <http://www.campaigner.com/>;, FuseMail =C2=AE <http://www.fusema=
il.com/>;, KeepItSafe =C2=AE <http://www.keepitsafe.com/> and Onebox =C2=AE =
<http://www.onebox.com/> are r egistered trademarks of j2 Global, Inc <http=
://www.j2.com/>. and its affiliates.
>>>>=20
>>>>=20
>>>>=20
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>>> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.or=
g/mailman/listinfo/users>
>>>=20
>>>=20
>>>=20
>>> --=20
>>> This message is subject to the CSIR's copyright terms and conditions,=
e-mail legal notice, and implemented Open Document Format (ODF) standard.=
=20
>>> The full disclaimer details can be found at
<http://www.csir.co.za/d=
isclaimer.html>http://www.csir.co.za/disclaimer.html
<http://www.csir.co.za=
/disclaimer.html>.
>>>=20
>>> This message has been scanned for viruses and dangerous content by Ma=
ilScanner <http://www.mailscanner.info/>,=20
>>> and is believed to be clean.
>>>=20
>>>=20
>>> Please consider the environment before printing this email.
>>>=20
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org=
/mailman/listinfo/users>
>>=20
>>=20
>> --=20
>> This message is subject to the CSIR's copyright terms and conditions, =
e-mail legal notice, and implemented Open Document Format (ODF) standard.=
=20
>> The full disclaimer details can be found at
<http://www.csir.co.za/di=
sclaimer.html>http://www.csir.co.za/disclaimer.html
<http://www.csir.co.za/=
disclaimer.html>.
>>=20
>> This message has been scanned for viruses and dangerous content by Mai=
lScanner <http://www.mailscanner.info/>,=20
>> and is believed to be clean.
>>=20
>>=20
>> Please consider the environment before printing this email.
>>=20
>=20
>=20
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
=20
=20
--=20
This message is subject to the CSIR's copyright terms and conditions, e-m=
ail
legal notice, and implemented Open Document Format (ODF) standard.=20
The full disclaimer details can be found at
http://www.csir.co.za/disclai=
mer.html.
=20
This message has been scanned for viruses and dangerous content by MailSc=
anner,=20
and is believed to be clean.
=20
Please consider the environment before printing this email.
=20
--PW0Eas8rCkcu1VkF
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBAgAGBQJXDMb9AAoJECXo5AApwsWzMjsQAMVh1jBmyG/6iYfL9V1grz9j
5i/fsIQB9kfEGeV5Jh8A1/o+DLnXLAwf8L0C1b4sW2nUQ//8nfrroDVrRaBqv/wL
vrzs5jcuzfTl56Zv6FGsYZDTrqVrACi997lKFkvzTiUOqH8hw9ctmVen+lzlCga7
yuYCb/SpeH/Nx5I5o2Gfnd54jd5FSdMZfqoCS6TVbVPtlFjbN/spBCT9VT4SpoIn
5RRGGbPqcd36BpWIyGp4iZ2BAtqfIZDcJWg2y+kbMFRWnIlJ5wkixtPvAlMIBC3d
ehyfjPRa2/aTIT24Xth/vDzAYL0S5phpnsvmlDiendxerz4eT+4PcP0TVws5NvD1
1tV1qNd4i7dnE8d7AlqjhnjEyMijIgjkF55BRj5PJ21CkTdwVsk5BcPmEP3lUH6c
HwwXZGPY5iC6Q+pJ0KERVutJDmp/OJ86SlTnRwinlHRTtjagklmcvkKyH44TgfaF
t+IE3/g6l0IgPkg4kU30ignUxaFvikH9Y6esnsHglzuVfAiMliR0hU7mLx0yrh5E
2fI80iV+alp7eRjkHEVZ7A/rKfOJWPY2y0+P6WnoKIkYQ+uNlZSOCERRJIv56XDO
66feEUuzxna/ucxBYyEiPmgNIx64PPaXr6eiA8cgG3TpLZgQ8E4Od6qP7MI+2ikR
W1Hy81Lq7KrQwGzgmZR8
=gQkK
-----END PGP SIGNATURE-----
--PW0Eas8rCkcu1VkF--