11:38 a.m.
--Apple-Mail=_F4D45E80-CC6B-492F-BC3F-6304354C342D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=windows-1252
Hi Ravi and colleagues
(apologies for hijacking this thread but I=92m not sure where else to repor=
t this (and it is related).)
With gluster 3.7.10, running
#gluster volume set <volname> group virt
fails with:
volume set: failed: option : eager-lock does not exist
Did you mean eager-lock?
I had to remove the eager-lock setting from /var/lib/glusterd/groups/virt t=
o get this to work. It seems like setting eager-lock has been removed from =
latest gluster. Is this correct? Either way, is there anything else I shoul=
d do?
Cheers,
Roderick
On 12 Feb 2016, at 6:18 AM, Ravishankar N
<ravishankar(a)redhat.com> wrote:
=20
Hi Bill,
Can you enable virt-profile setting for your volume and see if that helps=
? You
need to enable this optimization when you create the volume using ovr=
it, or use the following command for an existing volume:
=20
#gluster volume set <volname> group virt
=20
-Ravi
=20
=20
On 02/12/2016 05:22 AM, Bill James wrote:
> My apologies, I'm showing how much of a noob I am.
> Ignore last direct to gluster numbers, as that wasn't really glusterfs.
>=20
>=20
> [root@ovirt2 test ~]# mount -t glusterfs ovirt2-ks.test.j2noc.com:/gv1 /=
mnt/tmp/
> [root@ovirt2 test ~]# time dd if=3D/dev/zero
of=3D/mnt/tmp/testfile2 bs=
=3D1M count=3D1000 oflag=3Ddirect
> 1048576000 bytes (1.0 GB) copied, 65.8596 s, 15.9 MB/s
>=20
> That's more how I expected, it is pointing to glusterfs performance.
>=20
>=20
>=20
> On 02/11/2016 03:27 PM, Bill James wrote:
>> don't know if it helps, but I ran a few more tests, all from the same h=
ardware node.
>>=20
>> The VM:
>> [root@billjov1 ~]# time dd if=3D/dev/zero of=3D/root/testfile bs=3D1M c=
ount=3D1000 oflag=3Ddirect
>> 1048576000 bytes (1.0 GB) copied, 62.5535 s, 16.8 MB/s
>>=20
>> Writing directly to gluster volume:
>> [root@ovirt2 test ~]# time dd if=3D/dev/zero of=3D/gluster-store/brick1=
/gv1/testfile bs=3D1M count=3D1000 oflag=3Ddirect
>> 1048576000 bytes (1.0 GB) copied, 9.92048 s, 106 MB/s
>>=20
>>=20
>> Writing to NFS volume:
>> [root@ovirt2 test ~]# time dd if=3D/dev/zero of=3D/mnt/storage/qa/testf=
ile bs=3D1M count=3D1000 oflag=3Ddirect
>> 1048576000 bytes (1.0 GB) copied, 10.5776 s, 99.1 MB/s
>>=20
>> NFS & Gluster are using the same interface. Tests were not run at same =
time.
>>=20
>> This would suggest my problem isn't glusterfs, but the VM performance.
>>=20
>>=20
>>=20
>> On 02/11/2016 03:13 PM, Bill James wrote:
>>> xml attached.=20
>>>=20
>>>=20
>>> On 02/11/2016 12:28 PM, Nir Soffer wrote:=20
>>>> On Thu, Feb 11, 2016 at 8:27 PM, Bill James <bill.james(a)j2.com>
<mail=
to:bill.james@j2.com> wrote:=20
>>>>> thank you for the reply.=20
>>>>>=20
>>>>> We setup gluster using the names associated with NIC 2 IP.=20
>>>>> Brick1: ovirt1-ks.test.j2noc.com:/gluster-store/brick1/gv1=20
>>>>> Brick2: ovirt2-ks.test.j2noc.com:/gluster-store/brick1/gv1=20
>>>>> Brick3: ovirt3-ks.test.j2noc.com:/gluster-store/brick1/gv1=20
>>>>>=20
>>>>> That's NIC 2's IP.=20
>>>>> Using 'iftop -i eno2 -L 5 -t' :=20
>>>>>=20
>>>>> dd if=3D/dev/zero of=3D/root/testfile bs=3D1M count=3D1000 oflag=3Dd=
irect=20
>>>>> 1048576000 bytes (1.0 GB) copied, 68.0714 s, 15.4
MB/s=20
>>>> Can you share the xml of this vm? You can find it in vdsm log,=20
>>>> at the time you start the vm.=20
>>>>=20
>>>> Or you can do (on the host):=20
>>>>=20
>>>> # virsh=20
>>>> virsh # list=20
>>>> (username: vdsm@ovirt password: shibboleth)=20
>>>> virsh # dumpxml vm-id=20
>>>>=20
>>>>> Peak rate (sent/received/total): 281Mb 5.36Mb=
=20
>>>>> 282Mb=20
>>>>> Cumulative (sent/received/total): 1.96GB 14.6MB=
=20
>>>>> 1.97GB=20
>>>>>=20
>>>>> gluster volume info gv1:=20
>>>>> Options Reconfigured:=20
>>>>> performance.write-behind-window-size: 4MB=20
>>>>> performance.readdir-ahead: on=20
>>>>> performance.cache-size: 1GB=20
>>>>> performance.write-behind: off=20
>>>>>=20
>>>>> performance.write-behind: off didn't help.=20
>>>>> Neither did any other changes I've tried.=20
>>>>>=20
>>>>>=20
>>>>> There is no VM traffic on this VM right now except my test.=20
>>>>>=20
>>>>>=20
>>>>>=20
>>>>> On 02/10/2016 11:55 PM, Nir Soffer wrote:=20
>>>>>> On Thu, Feb 11, 2016 at 2:42 AM, Ravishankar N
<ravishankar(a)redhat.=
com> <mailto:ravishankar@redhat.com>=20
>>>>>> wrote:=20
>>>>>>> +gluster-users=20
>>>>>>>=20
>>>>>>> Does disabling 'performance.write-behind' give a
better throughput=
?=20
>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>>> On 02/10/2016 11:06 PM, Bill James wrote:=20
>>>>>>>> I'm setting up a ovirt cluster using glusterfs and
noticing not s=
tellar=20
>>>>>>>> performance.=20
>>>>>>>> Maybe my setup could use some adjustments?=20
>>>>>>>>=20
>>>>>>>> 3 hardware nodes running centos7.2, glusterfs 3.7.6.1,
ovirt 3.6.=
2.6-1.=20
>>>>>>>> Each node has 8 spindles configured
in 1 array which is split usi=
ng LVM=20
>>>>>>>> with one logical volume for system
and one for gluster.=20
>>>>>>>> They each have 4 NICs,=20
>>>>>>>> NIC1 =3D ovirtmgmt=20
>>>>>>>> NIC2 =3D gluster (1GbE)=20
>>>>>> How do you ensure that gluster trafic is using this nic?=20
>>>>>>=20
>>>>>>>> NIC3 =3D VM traffic=20
>>>>>> How do you ensure that vm trafic is using this nic?=20
>>>>>>=20
>>>>>>>> I tried with default glusterfs settings=20
>>>>>> And did you find any difference?=20
>>>>>>=20
>>>>>>>> and also with:=20
>>>>>>>> performance.cache-size: 1GB=20
>>>>>>>> performance.readdir-ahead: on=20
>>>>>>>> performance.write-behind-window-size: 4MB=20
>>>>>>>>=20
>>>>>>>> [root@ovirt3 test scripts]# gluster volume info gv1=20
>>>>>>>>=20
>>>>>>>> Volume Name: gv1=20
>>>>>>>> Type: Replicate=20
>>>>>>>> Volume ID: 71afc35b-09d7-4384-ab22-57d032a0f1a2=20
>>>>>>>> Status: Started=20
>>>>>>>> Number of Bricks: 1 x 3 =3D 3=20
>>>>>>>> Transport-type: tcp=20
>>>>>>>> Bricks:=20
>>>>>>>> Brick1:
ovirt1-ks.test.j2noc.com:/gluster-store/brick1/gv1=20
>>>>>>>> Brick2:
ovirt2-ks.test.j2noc.com:/gluster-store/brick1/gv1=20
>>>>>>>> Brick3:
ovirt3-ks.test.j2noc.com:/gluster-store/brick1/gv1=20
>>>>>>>> Options Reconfigured:=20
>>>>>>>> performance.cache-size: 1GB=20
>>>>>>>> performance.readdir-ahead: on=20
>>>>>>>> performance.write-behind-window-size: 4MB=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> Using simple dd test on VM in ovirt:=20
>>>>>>>> dd if=3D/dev/zero of=3D/root/testfile bs=3D1G
count=3D1 oflag=
=3Ddirect=20
>>>>>> block size of 1G?!=20
>>>>>>=20
>>>>>> Try 1M (our default for storage operations)=20
>>>>>>=20
>>>>>>>> 1073741824 bytes (1.1 GB) copied, 65.9337 s, 16.3
MB/s=20
>>>>>>>>=20
>>>>>>>> Another VM not in ovirt using nfs:=20
>>>>>>>> dd if=3D/dev/zero of=3D/root/testfile bs=3D1G
count=3D1 ofla=
g=3Ddirect=20
>>>>>>>> 1073741824 bytes (1.1 GB)
copied, 27.0079 s, 39.8 MB/s=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> Is that expected or is there a better way to set it up to
get bet=
ter=20
>>>>>>>> performance?=20
>>>>>> Adding Niels for advice.=20
>>>>>>=20
>>>>>>>> This email, its contents and ....=20
>>>>>> Please avoid this, this is a public mailing list, everything you
wr=
ite=20
>>>>>> here is public.=20
>>>>>>=20
>>>>>> Nir=20
>>>>> I'll have to look into how to remove this sig for this mailing
list.=
...=20
>>>>>=20
>>>>> Cloud Services for Business www.j2.com <http://www.j2.com/>=20
>>>>> j2 | eFax | eVoice | FuseMail | Campaigner | KeepItSafe | Onebox=20
>>>>>=20
>>>>>=20
>>>>> This email, its contents and attachments contain information from j2=
Global,=20
>>>>> Inc. and/or its affiliates which may be
privileged, confidential or=
=20
>>>>> otherwise protected from disclosure. The
information is intended to =
be for=20
>>>>> the addressee(s) only. If you are not an
addressee, any disclosure, =
copy,=20
>>>>> distribution, or use of the contents of this
message is prohibited. =
If you=20
>>>>> have received this email in error please notify
the sender by reply =
e-mail=20
>>>>> and delete the original message and any copies.
(c) 2015 j2 Global, =
Inc. All=20
>>>>> rights reserved. eFax, eVoice, Campaigner,
FuseMail, KeepItSafe, and=
Onebox=20
>>>>> are registered trademarks of j2 Global, Inc. and
its affiliates.=20
>>>=20
>>>=20
>>>=20
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/=
mailman/listinfo/users>
utm_campaign=3Demployeeemail>
>> This email, its contents and attachments contain information
from j2 Gl=
obal, Inc
<http://www.j2.com/?utm_source=3Dj2global&utm_medium=3Dxsell-refe=
rral&utm_campaign=3Demployemail>. and/or its affiliates which may be privil=
eged, confidential or otherwise protected from disclosure. The information =
is intended to be for the addressee(s) only. If you are not an addressee, a=
ny disclosure, copy, distribution, or use of the contents of this message i=
s prohibited. If you have received this email in error please notify the se=
nder by reply e-mail and delete the original message and any copies. =A9 20=
15 j2 Global, Inc <http://www.j2.com/>;. All rights reserved. eFax =AE <http=
://www.efax.com/>, eVoice =AE <http://www.evoice.com/>;, Campaigner =AE <htt=
p://www.campaigner.com/>;, FuseMail =AE <http://www.fusemail.com/>;, KeepItSa=
fe =AE <http://www.keepitsafe.com/> and Onebox =AE <http://www.onebox.com/>=
are r egistered trademarks of j2 Global, Inc <http://www.j2.com/>;. and its=
affiliates.
>>=20
>>=20
>>=20
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/m=
ailman/listinfo/users>
tm_campaign=3Demployeeemail>
> This email, its contents and attachments contain information from
j2 Glo=
bal, Inc
<http://www.j2.com/?utm_source=3Dj2global&utm_medium=3Dxsell-refer=
ral&utm_campaign=3Demployemail>. and/or its affiliates which may be privile=
ged, confidential or otherwise protected from disclosure. The information i=
s intended to be for the addressee(s) only. If you are not an addressee, an=
y disclosure, copy, distribution, or use of the contents of this message is=
prohibited. If you have received this email in error please notify the sen=
der by reply e-mail and delete the original message and any copies. =A9 201=
5 j2 Global, Inc <http://www.j2.com/>;. All rights reserved. eFax =AE <http:=
//www.efax.com/>, eVoice =AE <http://www.evoice.com/>;, Campaigner =AE <http=
://www.campaigner.com/>, FuseMail =AE <http://www.fusemail.com/>;, KeepItSaf=
e =AE <http://www.keepitsafe.com/> and Onebox =AE <http://www.onebox.com/> =
are r egistered trademarks of j2 Global, Inc <http://www.j2.com/>;. and its =
affiliates.
>=20
>=20
>=20
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/ma=
ilman/listinfo/users>
=20
=20
=20
--=20
This message is subject to the CSIR's copyright terms and conditions, e-m=
ail
legal notice, and implemented Open Document Format (ODF) standard.=20
The full disclaimer details can be found at
http://www.csir.co.za/disclai=
mer.html
<http://www.csir.co.za/disclaimer.html>;.
=20
This message has been scanned for viruses and dangerous content by MailSc=
anner
<http://www.mailscanner.info/>,=20
and is believed to be clean.
=20
=20
Please consider the environment before printing this email.
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--=20
This message is subject to the CSIR's copyright terms and conditions, e-mai=
l legal notice, and implemented Open Document Format (ODF) standard.=20
The full disclaimer details can be found at http://www.csir.co.za/disclaime=
r.html.
This message has been scanned for viruses and dangerous content by MailScan=
ner,=20
and is believed to be clean.
Please consider the environment before printing this email.
--Apple-Mail=_F4D45E80-CC6B-492F-BC3F-6304354C342D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=windows-1252
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html charset=
=3Dwindows-1252"></head><body style=3D"word-wrap: break-word;
-webkit-nbsp-=
mode: space; -webkit-line-break: after-white-space;" class=3D"">Hi Ravi
and=
colleagues<div class=3D""><br
class=3D""></div><div class=3D"">(apologies =
for hijacking this thread but I=92m not sure where else to report this (and=
it is related).)<div class=3D""><br
class=3D""></div><div class=3D"">With =
gluster 3.7.10, running</div><div class=3D"">#gluster volume set
<volnam=
e> group virt</div><div class=3D"">fails
with:</div><div class=3D"">volu=
me set: failed: option : eager-lock does not exist</div><div
class=3D"">Did=
you mean eager-lock?</div><div class=3D""><br
class=3D""></div><div class=
=3D"">I had to remove the eager-lock setting from /var/lib/glusterd/groups/=
virt to get this to work. It seems like setting eager-lock has been removed=
from latest gluster. Is this correct? Either way, is there anything else I=
should do?</div><div class=3D""><br
class=3D""></div><div class=3D"">Cheer=
s,</div><div class=3D""><br
class=3D""></div><div class=3D"">Roderick<br cl=
ass=3D""><div class=3D""><br
class=3D""><div><blockquote type=3D"cite" clas=
s=3D""><div class=3D"">On 12 Feb 2016, at 6:18 AM,
Ravishankar N <<a hre=
f=3D"mailto:ravishankar@redhat.com"
class=3D"">ravishankar(a)redhat.com</a>&g=
t; wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D"">
=20=20
<meta content=3D"text/html; charset=3Dwindows-1252"
http-equiv=3D"Conte=
nt-Type" class=3D"">
=20=20
<div text=3D"#000000" bgcolor=3D"#FFFFFF"
class=3D"">
<div class=3D"moz-cite-prefix">Hi Bill,<br
class=3D"">
Can you enable virt-profile setting for your volume and see if
that helps? You need to enable this optimization when you create
the volume using ovrit, or use the following command for an
existing volume:<br class=3D"">
<tt class=3D""><br class=3D"">
#gluster volume set <volname> group virt</tt><br
class=3D"">
<br class=3D"">
-Ravi<br class=3D"">
<br class=3D"">
<br class=3D"">
On 02/12/2016 05:22 AM, Bill James wrote:<br class=3D"">
</div>
<blockquote cite=3D"mid:56BD1EB9.5000802@j2.com" type=3D"cite"
class=3D=
"">
<meta content=3D"text/html; charset=3Dwindows-1252"
http-equiv=3D"Con=
tent-Type" class=3D"">
My apologies, I'm showing how much of a noob I am.<br
class=3D"">
Ignore last direct to gluster numbers, as that wasn't really
glusterfs.<br class=3D"">
<br class=3D"">
<br class=3D"">
[root@ovirt2 test ~]# mount -t glusterfs
<a href=3D"http://ovirt2-ks.test.j2noc.com"
class=3D"">ovirt2-ks.test=
.j2noc.com</a>:/gv1 /mnt/tmp/<br class=3D"">
[root@ovirt2 test ~]# time dd if=3D/dev/zero of=3D/mnt/tmp/testfile2
bs=3D1M count=3D1000 oflag=3Ddirect<br class=3D"">
1048576000 bytes (1.0 GB) copied, 65.8596 s, 15.9 MB/s<br
class=3D"">
<br class=3D"">
That's more how I expected, it is pointing to glusterfs
performance.<br class=3D"">
<br class=3D"">
<br class=3D"">
<br class=3D"">
<div class=3D"moz-cite-prefix">On 02/11/2016 03:27 PM, Bill James
wrote:<br class=3D"">
</div>
<blockquote cite=3D"mid:56BD18D7.8080909@j2.com"
type=3D"cite" class=
=3D"">
<meta http-equiv=3D"Content-Type" content=3D"text/html;
charset=3Dwindows-1252" class=3D"">
don't know if it helps, but I ran a few more tests, all from the
same hardware node.<br class=3D"">
<br class=3D"">
The VM:<br class=3D"">
[root@billjov1 ~]# time dd if=3D/dev/zero of=3D/root/testfile bs=3D=
1M
count=3D1000 oflag=3Ddirect<br class=3D"">
1048576000 bytes (1.0 GB) copied, 62.5535 s, 16.8 MB/s<br class=3D"=
">
<br class=3D"">
Writing directly to gluster volume:<br class=3D"">
[root@ovirt2 test ~]# time dd if=3D/dev/zero
of=3D/gluster-store/brick1/gv1/testfile bs=3D1M count=3D1000
oflag=3Ddirect<br class=3D"">
1048576000 bytes (1.0 GB) copied, 9.92048 s, 106 MB/s<br
class=3D"">
<br class=3D"">
<br class=3D"">
Writing to NFS volume:<br class=3D"">
[root@ovirt2 test ~]# time dd if=3D/dev/zero
of=3D/mnt/storage/qa/testfile bs=3D1M count=3D1000 oflag=3Ddirect<b=
r class=3D"">
1048576000 bytes (1.0 GB) copied, 10.5776 s, 99.1 MB/s<br class=3D"=
">
<br class=3D"">
NFS & Gluster are using the same interface. Tests were not
run at same time.<br class=3D"">
<br class=3D"">
This would suggest my problem isn't glusterfs, but the VM
performance.<br class=3D"">
<br class=3D"">
<br class=3D"">
<br class=3D"">
<div class=3D"moz-cite-prefix">On 02/11/2016 03:13 PM, Bill James
wrote:<br class=3D"">
</div>
<blockquote cite=3D"mid:56BD1589.8090506@j2.com"
type=3D"cite" clas=
s=3D"">xml
attached. <br class=3D"">
<br class=3D"">
<br class=3D"">
On 02/11/2016 12:28 PM, Nir Soffer wrote: <br class=3D"">
<blockquote type=3D"cite" class=3D"">On Thu, Feb 11,
2016 at 8:27=
PM, Bill
James <a moz-do-not-send=3D"true"
class=3D"moz-txt-link-rfc2396=
E"
href=3D"mailto:bill.james@j2.com"><bill.james@j2.com></a>
wrote: <br class=3D"">
<blockquote type=3D"cite" class=3D"">thank you for
the reply. <=
br class=3D"">
<br class=3D"">
We setup gluster using the names associated with NIC 2
IP. <br class=3D"">
Brick1:
<a href=3D"http://ovirt1-ks.test.j2noc.com"
class=3D"">ovirt1=
-ks.test.j2noc.com</a>:/gluster-store/brick1/gv1 <br class=3D"">
Brick2:
<a href=3D"http://ovirt2-ks.test.j2noc.com"
class=3D"">ovirt2=
-ks.test.j2noc.com</a>:/gluster-store/brick1/gv1 <br class=3D"">
Brick3:
<a href=3D"http://ovirt3-ks.test.j2noc.com"
class=3D"">ovirt3=
-ks.test.j2noc.com</a>:/gluster-store/brick1/gv1 <br class=3D"">
<br class=3D"">
That's NIC 2's IP. <br class=3D"">
Using 'iftop -i eno2 -L 5 -t' : <br class=3D"">
<br class=3D"">
dd if=3D/dev/zero of=3D/root/testfile bs=3D1M count=3D1000
oflag=3Ddirect <br class=3D"">
1048576000 bytes (1.0 GB) copied, 68.0714 s, 15.4 MB/s <br cl=
ass=3D"">
</blockquote>
Can you share the xml of this vm? You can find it in vdsm
log, <br class=3D"">
at the time you start the vm. <br class=3D"">
<br class=3D"">
Or you can do (on the host): <br class=3D"">
<br class=3D"">
# virsh <br class=3D"">
virsh # list <br class=3D"">
(username: vdsm@ovirt password: shibboleth) <br class=3D"">
virsh # dumpxml vm-id <br class=3D"">
<br class=3D"">
<blockquote type=3D"cite" class=3D"">Peak rate
(sent/received/total): &nb=
sp; =
281Mb 5.36Mb <br class=3D"">
282Mb <br class=3D"">
Cumulative
(sent/received/total): &nbs=
p; &=
nbsp;
1.96GB 14.6MB <br class=3D"">
1.97GB <br class=3D"">
<br class=3D"">
gluster volume info gv1: <br class=3D"">
Options Reconfigured: <br class=3D"">
performance.write-behind-window-size: 4MB <br class=3D=
"">
performance.readdir-ahead: on <br class=3D"">
performance.cache-size: 1GB <br class=3D"">
performance.write-behind: off <br class=3D"">
<br class=3D"">
performance.write-behind: off didn't help. <br
class=3D"">
Neither did any other changes I've tried. <br
class=3D"">
<br class=3D"">
<br class=3D"">
There is no VM traffic on this VM right now except my
test. <br class=3D"">
<br class=3D"">
<br class=3D"">
<br class=3D"">
On 02/10/2016 11:55 PM, Nir Soffer wrote: <br class=3D"">
<blockquote type=3D"cite" class=3D"">On Thu, Feb
11, 2016 at =
2:42 AM,
Ravishankar N <a moz-do-not-send=3D"true"
class=3D"moz-txt-=
link-rfc2396E"
href=3D"mailto:ravishankar@redhat.com"><ravishankar@redha=
t.com></a>
<br class=3D"">
wrote: <br class=3D"">
<blockquote type=3D"cite"
class=3D"">+gluster-users <br cla=
ss=3D"">
<br class=3D"">
Does disabling 'performance.write-behind' give a
better throughput? <br class=3D"">
<br class=3D"">
<br class=3D"">
<br class=3D"">
On 02/10/2016 11:06 PM, Bill James wrote: <br
class=3D"">
<blockquote type=3D"cite" class=3D"">I'm
setting up a ovi=
rt cluster
using glusterfs and noticing not stellar <br
class=3D"">
performance. <br class=3D"">
Maybe my setup could use some adjustments? <br class=3D=
"">
<br class=3D"">
3 hardware nodes running centos7.2, glusterfs
3.7.6.1, ovirt 3.6.2.6-1. <br class=3D"">
Each node has 8 spindles configured in 1 array which
is split using LVM <br class=3D"">
with one logical volume for system and one for
gluster. <br class=3D"">
They each have 4 NICs, <br class=3D"">
NIC1 =3D ovirtmgmt <br
class=3D"">
NIC2 =3D gluster (1GbE) <br
class=3D=
"">
</blockquote>
</blockquote>
How do you ensure that gluster trafic is using this nic?
<br class=3D"">
<br class=3D"">
<blockquote type=3D"cite" class=3D"">
<blockquote type=3D"cite"
class=3D""> NIC3 =
=3D VM traffic <br class=3D"">
</blockquote>
</blockquote>
How do you ensure that vm trafic is using this nic? <br cla=
ss=3D"">
<br class=3D"">
<blockquote type=3D"cite" class=3D"">
<blockquote type=3D"cite" class=3D"">I tried
with default=
glusterfs
settings <br class=3D"">
</blockquote>
</blockquote>
And did you find any difference? <br class=3D"">
<br class=3D"">
<blockquote type=3D"cite" class=3D"">
<blockquote type=3D"cite" class=3D"">and also
with: <br c=
lass=3D"">
performance.cache-size: 1GB <br class=3D"">
performance.readdir-ahead: on <br class=3D"">
performance.write-behind-window-size: 4MB <br class=3D"=
">
<br class=3D"">
[root@ovirt3 test scripts]# gluster volume info gv1
<br class=3D"">
<br class=3D"">
Volume Name: gv1 <br class=3D"">
Type: Replicate <br class=3D"">
Volume ID: 71afc35b-09d7-4384-ab22-57d032a0f1a2 <br cla=
ss=3D"">
Status: Started <br class=3D"">
Number of Bricks: 1 x 3 =3D 3 <br class=3D"">
Transport-type: tcp <br class=3D"">
Bricks: <br class=3D"">
Brick1:
<a href=3D"http://ovirt1-ks.test.j2noc.com"
class=3D"">=
ovirt1-ks.test.j2noc.com</a>:/gluster-store/brick1/gv1 <br
class=3D"">
Brick2:
<a href=3D"http://ovirt2-ks.test.j2noc.com"
class=3D"">=
ovirt2-ks.test.j2noc.com</a>:/gluster-store/brick1/gv1 <br
class=3D"">
Brick3:
<a href=3D"http://ovirt3-ks.test.j2noc.com"
class=3D"">=
ovirt3-ks.test.j2noc.com</a>:/gluster-store/brick1/gv1 <br
class=3D"">
Options Reconfigured: <br class=3D"">
performance.cache-size: 1GB <br class=3D"">
performance.readdir-ahead: on <br class=3D"">
performance.write-behind-window-size: 4MB <br class=3D"=
">
<br class=3D"">
<br class=3D"">
Using simple dd test on VM in ovirt: <br class=3D"">
dd if=3D/dev/zero of=3D/root/testfil=
e bs=3D1G count=3D1
oflag=3Ddirect <br class=3D"">
</blockquote>
</blockquote>
block size of 1G?! <br class=3D"">
<br class=3D"">
Try 1M (our default for storage operations) <br
class=3D"">
<br class=3D"">
<blockquote type=3D"cite" class=3D"">
<blockquote type=3D"cite"
class=3D""> &n=
bsp; 1073741824 bytes (1.1
GB) copied, 65.9337 s, 16.3 MB/s <br class=3D"">
<br class=3D"">
Another VM not in ovirt using nfs: <br class=3D"">
dd if=3D/dev/zero
of=3D/root/t=
estfile bs=3D1G count=3D1
oflag=3Ddirect <br class=3D"">
1073741824 b=
ytes (1.1 GB) copied, 27.0079 s,
39.8 MB/s <br class=3D"">
<br class=3D"">
<br class=3D"">
Is that expected or is there a better way to set it
up to get better <br class=3D"">
performance? <br class=3D"">
</blockquote>
</blockquote>
Adding Niels for advice. <br class=3D"">
<br class=3D"">
<blockquote type=3D"cite" class=3D"">
<blockquote type=3D"cite" class=3D"">This
email, its cont=
ents and
.... <br class=3D"">
</blockquote>
</blockquote>
Please avoid this, this is a public mailing list,
everything you write <br class=3D"">
here is public. <br class=3D"">
<br class=3D"">
Nir <br class=3D"">
</blockquote>
I'll have to look into how to remove this sig for this
mailing list.... <br class=3D"">
<br class=3D"">
Cloud Services for Business <a moz-do-not-send=3D"true" class=
=3D"moz-txt-link-abbreviated"
href=3D"http://www.j2.com/">www.j2.com</a> <b=
r class=3D"">
j2 | eFax | eVoice | FuseMail | Campaigner | KeepItSafe |
Onebox <br class=3D"">
<br class=3D"">
<br class=3D"">
This email, its contents and attachments contain
information from j2 Global, <br class=3D"">
Inc. and/or its affiliates which may be privileged,
confidential or <br class=3D"">
otherwise protected from disclosure. The information is
intended to be for <br class=3D"">
the addressee(s) only. If you are not an addressee, any
disclosure, copy, <br class=3D"">
distribution, or use of the contents of this message is
prohibited. If you <br class=3D"">
have received this email in error please notify the sender
by reply e-mail <br class=3D"">
and delete the original message and any copies. (c) 2015
j2 Global, Inc. All <br class=3D"">
rights reserved. eFax, eVoice, Campaigner, FuseMail,
KeepItSafe, and Onebox <br class=3D"">
are registered trademarks of j2 Global, Inc. and its
affiliates. <br class=3D"">
</blockquote>
</blockquote>
<br class=3D"">
<br class=3D"">
<fieldset class=3D"mimeAttachmentHeader"></fieldset>
<br class=3D"">
<pre wrap=3D""
class=3D"">_______________________________________=
________
Users mailing list
<a moz-do-not-send=3D"true" class=3D"moz-txt-link-abbreviated"
href=3D"mail=
to:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send=3D"true" class=3D"moz-txt-link-freetext"
href=3D"http://=
lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/ma...
info/users</a>
</pre>
</blockquote>
<br class=3D""><p class=3D""><a
moz-do-not-send=3D"true" href=3D"ht=
tp://www.j2.com/?utm_source=3Dj2global&utm_medium=3Dxsell-referra...
utm_campaign=3Demployeeemail" class=3D""><span
style=3D"color:windowtext; t=
ext-decoration:none" class=3D""><img
moz-do-not-send=3D"true" src=3D"http:/=
/home.j2.com/j2_Global_Cloud_Services/j2_Global_Email_Footer.jpg" alt=3D"ww=
w.j2.com" height=3D"46" width=3D"391" border=3D"0"
class=3D""></span></a></=
p><p class=3D""><span
style=3D"font-size:8.0pt;font-family:"Arial"=
;,"sans-serif";color:gray" class=3D"">This
email, its contents and attachments contain information from
<a moz-do-not-send=3D"true"
href=3D"http://www.j2.com/?utm_sour=
ce=3Dj2global&utm_medium=3Dxsell-referral&utm_campaign=3Demployemai=
l" class=3D"">j2
Global, Inc</a>. and/or its affiliates which may be
privileged, confidential or otherwise protected from
disclosure. The information is intended to be for the
addressee(s) only. If you are not an addressee, any
disclosure, copy, distribution, or use of the contents of
this message is prohibited. If you have received this email
in error please notify the sender by reply e-mail and delete
the original message and any copies. =A9 2015 <a moz-do-not-sen=
d=3D"true" href=3D"http://www.j2.com/" class=3D"">j2
Global, Inc</a>. All rights reserved. <a
moz-do-not-send=3D"t=
rue" href=3D"http://www.efax.com/" class=3D"">eFax
=AE</a>,
<a moz-do-not-send=3D"true"
href=3D"http://www.evoice.com/" cla=
ss=3D"">eVoice
=AE</a>, <a moz-do-not-send=3D"true"
href=3D"http://www.campa=
igner.com/" class=3D"">Campaigner =AE</a>, <a
moz-do-not-send=3D"true" href=
=3D"http://www.fusemail.com/" class=3D"">FuseMail
=AE</a>, <a moz-do-not-send=3D"true"
href=3D"http://www.keepi=
tsafe.com/" class=3D"">KeepItSafe =AE</a> and <a
moz-do-not-send=3D"true" h=
ref=3D"http://www.onebox.com/" class=3D"">Onebox
=AE</a> are r egistered trademarks of <a
moz-do-not-send=3D"t=
rue" href=3D"http://www.j2.com/" class=3D"">j2
Global, Inc</a>. and its affiliates.</span></p>
<br class=3D"">
<fieldset class=3D"mimeAttachmentHeader"></fieldset>
<br class=3D"">
<pre wrap=3D""
class=3D"">_________________________________________=
______
Users mailing list
<a moz-do-not-send=3D"true" class=3D"moz-txt-link-abbreviated"
href=3D"mail=
to:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send=3D"true" class=3D"moz-txt-link-freetext"
href=3D"http://=
lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/ma...
info/users</a>
</pre>
</blockquote>
<br class=3D""><p class=3D""><a
moz-do-not-send=3D"true" href=3D"http=
://www.j2.com/?utm_source=3Dj2global&utm_medium=3Dxsell-referral&ut=
m_campaign=3Demployeeemail" class=3D""><span
style=3D"color:windowtext;
text-decoration:none" class=3D""><img
moz-do-not-send=3D"true" =
src=3D"http://home.j2.com/j2_Global_Cloud_Services/j2_Global_Email_F...
pg" alt=3D"www.j2.com" height=3D"46" width=3D"391"
border=3D"0" class=3D"">=
</span></a></p><p class=3D""><span
style=3D"font-size:8.0pt;font-family:&qu=
ot;Arial","sans-serif";color:gray"
class=3D"">This
email, its contents and attachments contain information from <a m=
oz-do-not-send=3D"true"
href=3D"http://www.j2.com/?utm_source=3Dj2global&am=
p;utm_medium=3Dxsell-referral&utm_campaign=3Demployemail"
class=3D"">j2
Global, Inc</a>. and/or its affiliates which may be
privileged, confidential or otherwise protected from
disclosure. The information is intended to be for the
addressee(s) only. If you are not an addressee, any
disclosure, copy, distribution, or use of the contents of this
message is prohibited. If you have received this email in
error please notify the sender by reply e-mail and delete the
original message and any copies. =A9 2015 <a moz-do-not-send=3D"t=
rue" href=3D"http://www.j2.com/" class=3D"">j2 Global,
Inc</a>. All rights reserved. <a moz-do-not-send=3D"true"
href=
=3D"http://www.efax.com/" class=3D"">eFax =AE</a>, <a
moz-do-not-send=3D"tr=
ue" href=3D"http://www.evoice.com/" class=3D"">eVoice
=AE</a>, <a moz-do-not-send=3D"true"
href=3D"http://www.campaig=
ner.com/" class=3D"">Campaigner =AE</a>, <a
moz-do-not-send=3D"true" href=
=3D"http://www.fusemail.com/" class=3D"">FuseMail
=AE</a>, <a moz-do-not-send=3D"true"
href=3D"http://www.keepits=
afe.com/" class=3D"">KeepItSafe =AE</a> and <a
moz-do-not-send=3D"true" hre=
f=3D"http://www.onebox.com/" class=3D"">Onebox
=AE</a> are r egistered trademarks of <a
moz-do-not-send=3D"tru=
e" href=3D"http://www.j2.com/" class=3D"">j2 Global,
Inc</a>. and its affiliates.</span></p>
<br class=3D"">
<fieldset class=3D"mimeAttachmentHeader"></fieldset>
<br class=3D"">
<pre wrap=3D""
class=3D"">___________________________________________=
____
Users mailing list
<a class=3D"moz-txt-link-abbreviated"
href=3D"mailto:Users@ovirt.org">Users=
@ovirt.org</a>
<a class=3D"moz-txt-link-freetext"
href=3D"http://lists.ovirt.org/mailman/l=
istinfo/users">http://lists.ovirt.org/mailman/listinfo/users</...
</pre>
</blockquote>
<br class=3D"">
<br class=3D"">
<font face=3D"Verdana,Arial,Helvetica,Trebuchet MS" size=3D"1"
class=3D"">
<br class=3D"">--=20
<br class=3D"">This message is subject to the CSIR's copyright terms
and co=
nditions, e-mail legal notice, and implemented Open Document Format (ODF) s=
tandard.
<br class=3D"">The full disclaimer details can be found at <a
href=3D"http:=
//www.csir.co.za/disclaimer.html"
class=3D"">http://www.csir.co.za/disclaim=
er.html</a>.
<p class=3D"">
<br class=3D"">This message has been scanned for viruses and dangerous
cont=
ent by <a href=3D"http://www.mailscanner.info/" class=3D""><b
class=3D"">Ma=
ilScanner</b></a>,=20
<br class=3D"">and is believed to be clean.
</p></font><p class=3D""><font
face=3D"Verdana,Arial,Helvetica,Trebuchet MS=
" size=3D"1" class=3D"">
<br class=3D"">Please consider the environment before printing this
email.
</font>
</p></div>
_______________________________________________<br class=3D"">Users
mailing=
list<br class=3D""><a href=3D"mailto:Users@ovirt.org"
class=3D"">Users@ovi=
rt.org</a><br
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br c=
lass=3D""></div></blockquote></div><br
class=3D""></div></div></div><font f=
ace=3D"Verdana,Arial,Helvetica,Trebuchet MS" size=3D"1">
<br />--=20
<br />This message is subject to the CSIR's copyright terms and conditions,=
e-mail legal notice, and implemented Open Document Format (ODF) standard.
<br />The full disclaimer details can be found at <a
href=3D"http://www.csi=
r.co.za/disclaimer.html">http://www.csir.co.za/disclaimer.html</a>.
<p>
<br />This message has been scanned for viruses and dangerous content by <a=
href=3D"http://www.mailscanner.info/"><b>MailScanner</b></a>,=20
<br />and is believed to be clean.
<p>
<br />Please consider the environment before printing this email.
</font>
</body></html>
--Apple-Mail=_F4D45E80-CC6B-492F-BC3F-6304354C342D--