Re: [ovirt-users] gluster performance oVirt 3.4

--Alternative_=_Boundary_=_1399919906 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On the Gluster site there is a QEMU repo I'll send you the link latter toda= y. But essentially CentOS doesn't include the Gluster client or libraries s= o they can't compile against it.<br>I'm not sure why CentOS and Scientific = Linux don't include it but I think it might just be an over site since Glus= ter 3.4 is new set of packages which were added in the latest release of RH= EL 6.<br>In truth you can simply unpack the source RPM and rebuild it with = Gluster support its not difficult but I don't remember the flags you need t= o pass the rpmbuild command off the top of my head. That said I'd probably = use the ones off the Gluster site any&nbsp;way.&nbsp;<br>If you are using G= luster 3.5 the ones included in RHEL are incompatible because Gluster still= has large API changes between minor releases and they were compiled agains= t 3.4.&nbsp;<br><br>Also Gluster 3.5 is a brand new release so I wouldn't r= ule out the possibility of a&nbsp;bug. So&nbsp;if the Gluster enabled QEMU = rpms don't help you may need to inquire on the Gluster mailing list.<br><br= pan id=3D"signature"><div style=3D"font-family: arial, sans-serif; font-siz= e: 12px;color: #999999;">-- Sent from my HP Pre3</div><br></span><span styl= e=3D"color:navy; font-family:Prelude, Verdana, san-serif; "><hr align=3D"le= ft" style=3D"width:75%">On May 12, 2014 8:58, Tobias Honacker &lt;tobias@ho= nacker.info&gt; wrote: <br><br></span><div dir=3D"ltr">I&#39;ve got the sam= e issue using the same versions as Korsaks, using CentOS 6.5, too.<div>I co= uld launch vms with gluster=C2=A0<span style=3D"color:rgb(51,51,51);font-fa= mily:arial,sans-serif;font-size:13px;line-height:17px">block backend driver= =C2=A0using the standard package of libvirt from centos repo.</span></div <div><font color=3D"#333333" face=3D"arial, sans-serif"><span style=3D"line= -height:17px">but ovirt does not run the VM with the qemu=C2=A0</span></fon= t><span style=3D"color:rgb(0,0,0);font-family:Helvetica,Arial,sans-serif;fo= nt-size:12px;line-height:18px">GFAPI integration, the path of the disk usin= g the fuse mount.</span></div <div><font color=3D"#000000" face=3D"Helvetica, Arial, sans-serif"><span st= yle=3D"font-size:12px;line-height:18px"><br></span></font></div><div><font = color=3D"#000000" face=3D"Helvetica, Arial, sans-serif"><span style=3D"font= -size:12px;line-height:18px"><div &lt;disk type=3D&#39;file&#39; device=3D&#39;disk&#39; snapshot=3D&#39;no&#= 39;&gt;</div><div>=C2=A0 =C2=A0 =C2=A0 &lt;driver name=3D&#39;qemu&#39; typ= e=3D&#39;raw&#39; cache=3D&#39;none&#39; error_policy=3D&#39;stop&#39; io= =3D&#39;threads&#39;/&gt;</div <div>=C2=A0 =C2=A0 =C2=A0 &lt;source file=3D&#39;/rhev/data-center/mnt/glus= terSD/localhost:VMDATA/0add493f-0a7f-4b32-bcd5-ff25ca504b8b/images/68dbbc67= -ea24-45a9-8727-4f85d100d1bb/8fe386f7-2aeb-43c2-bcb0-f76829c876b4&#39;&gt;<= /div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;seclabel model=3D&#39;selinux&#39= ; relabel=3D&#39;no&#39;/&gt;</div <div>=C2=A0 =C2=A0 =C2=A0 &lt;/source&gt;</div><div>=C2=A0 =C2=A0 =C2=A0 &l= t;target dev=3D&#39;vda&#39; bus=3D&#39;virtio&#39;/&gt;</div><div>=C2=A0 = =C2=A0 =C2=A0 &lt;serial&gt;&lt;/serial&gt;</div><div>=C2=A0 =C2=A0 =C2=A0 = &lt;alias name=3D&#39;virtio-disk0&#39;/&gt;</div><div>=C2=A0 =C2=A0 =C2=A0= &lt;address type=3D&#39;pci&#39; domain=3D&#39;0x0000&#39; bus=3D&#39;0x00= &#39; slot=3D&#39;0x06&#39; function=3D&#39;0x0&#39;/&gt;</div <div>=C2=A0 =C2=A0 &lt;/disk&gt;</div></span></font><div><br></div></div><d= iv><br></div><div>or am i wrong?</div><div class=3D"gmail_extra"><br><br><d= iv class=3D"gmail_quote">On Mon, May 12, 2014 at 7:41 AM, Vadims Korsaks <s= pan dir=3D"ltr">&lt;<a href=3D"mailto:truckk@inbox.lv" target=3D"_blank">tr= uckk(a)inbox.lv&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex">Underlying FS is XFS<brGlusterFS - glusterfs-3.5.0-2.el6<brI&#39;m using CentOS, if this is problem could RHEL<brpackages be used? why CentOS packages are compiled<brwithout native glusterfs support?<br<brCit=C4=93jot Paul Robert Marino &lt;<a href=3D"mailto:prmarino1@gmail.com" = target=3D&quot;_blank&quot;&gt;prmarino1(a)gmail.com&lt;/a&gt;&amp;gt; :<br<div>&gt; =C2=A0What&#39;s the underlying filesystem for gluster is<brit XFS?<br&gt; =C2=A0What version of gluster are you using?<br&gt; =C2=A0What distro are you using and if its not RHEL<bror Fedora are you using a version of QEMU with<brgluster support compiled in keep in mind the<brversions with CentOS and Scientific Linux do not<brinclude Gluster native support compiled in.<br&gt;<br&gt;<br&gt;<br&gt; =C2=A0-- Sent from my HP Pre3<br&gt;<br</div><div><div>&gt; =C2=A0 On May 11, 2014 5:40, Vadims Korsaks<br&lt;<a href=3D"mailto:truckk@inbox.lv" target=3D&quot;_blank&quot;&gt;truckk(a)inbox.lv&lt;/a= &gt;<br&gt; =C2=A0Cit=C4=93jot Vijay Bellur &lt;<a href=3D"mailto:vbellur@redhat.c= om" target=3D&quot;_blank&quot;&gt;vbellur(a)redhat.com&lt;/a&gt;&amp;gt; :<br&gt; =C2=A0&gt; On 05/11/2014 02:04 AM, Vadims Korsaks wrote:<br&gt; =C2=A0&gt; &gt; HI!<br&gt; =C2=A0&gt; &gt;<br&gt; =C2=A0&gt; &gt; Created 2 node setup with oVirt 3.4 and<br&gt; =C2=A0CentOS 6.5, for storage created<br&gt; =C2=A0&gt; &gt; 2 node replicated gluster (3.5) fs on same<br&gt; =C2=A0hosts with oVirt.<br&gt; =C2=A0&gt; &gt; mount looks like this:<br&gt; =C2=A0&gt; &gt; 127.0.0.1:/gluster01 on<br&gt; =C2=A0&gt; &gt;<br&gt;<br/rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01<br&gt; =C2=A0type fuse.glusterfs<br&gt; =C2=A0&gt; &gt;<br&gt;<br(rw,default_permissions,allow_other,max_read=3D131072)<br&gt; =C2=A0&gt; &gt;<br&gt; =C2=A0&gt; &gt; when i making gluster test with dd, something<br&gt; =C2=A0like<br&gt; =C2=A0&gt; &gt; dd if=3D/dev/zero bs=3D1M count=3D20000<br&gt; =C2=A0&gt; &gt;<br&gt;<brof=3D/rhev/data-center/mnt/glusterSD/<a href=3D"http://127.0.0.1" target=3D= "_blank">127.0.0.1</a>\:_gluster01/kaka<br&gt; =C2=A0&gt; &gt; i&#39;m gettting speed ~ 110 MB/s, so this is<br&gt; =C2=A01Gbps speed of ethernet adapter<br&gt; =C2=A0&gt; &gt;<br&gt; =C2=A0&gt; &gt; but with in VM created in oVirt speed is<br&gt; =C2=A0lower than 20 MB/s<br&gt; =C2=A0&gt; &gt;<br&gt; =C2=A0&gt; &gt; why there is so huge difference?<br&gt; =C2=A0&gt; &gt; how can improve VMs disks speed?<br&gt; =C2=A0&gt; &gt;<br&gt; =C2=A0&gt;<br&gt; =C2=A0&gt; What are your gluster volume settings? Have you<br&gt; =C2=A0applied the following<br&gt; =C2=A0&gt; performance tunables in gluster&#39;s virt profile:<br&gt; =C2=A0&gt;<br&gt; =C2=A0&gt; eager-lock=3Denable<br&gt; =C2=A0&gt; remote-dio=3Denable<br&gt; =C2=A0&gt;<br&gt; =C2=A0&gt; Regards,<br&gt; =C2=A0&gt; Vijay<br&gt; =C2=A0&gt;<br&gt; =C2=A0setting were:<br&gt; =C2=A0[root@centos155 ~]# gluster volume info gluster01<br&gt;<br&gt; =C2=A0Volume Name: gluster01<br&gt; =C2=A0Type: Replicate<br&gt; =C2=A0Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6<br&gt; =C2=A0Status: Started<br&gt; =C2=A0Number of Bricks: 1 x 2 =3D 2<br&gt; =C2=A0Transport-type: tcp<br&gt; =C2=A0Bricks:<br&gt; =C2=A0Brick1: 10.2.75.152:/mnt/gluster01/brick<br&gt; =C2=A0Brick2: 10.2.75.155:/mnt/gluster01/brick<br&gt; =C2=A0Options Reconfigured:<br&gt; =C2=A0storage.owner-gid: 36<br&gt; =C2=A0storage.owner-uid: 36<br&gt;<br&gt;<br&gt; =C2=A0add your settings settings now it looks<br&gt;<br&gt; =C2=A0[root@centos155 ~]# gluster volume info gluster01<br&gt;<br&gt; =C2=A0Volume Name: gluster01<br&gt; =C2=A0Type: Replicate<br&gt; =C2=A0Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6<br&gt; =C2=A0Status: Started<br&gt; =C2=A0Number of Bricks: 1 x 2 =3D 2<br&gt; =C2=A0Transport-type: tcp<br&gt; =C2=A0Bricks:<br&gt; =C2=A0Brick1: 10.2.75.152:/mnt/gluster01/brick<br&gt; =C2=A0Brick2: 10.2.75.155:/mnt/gluster01/brick<br&gt; =C2=A0Options Reconfigured:<br&gt; =C2=A0network.remote-dio: enable<br&gt; =C2=A0cluster.eager-lock: enable<br&gt; =C2=A0storage.owner-gid: 36<br&gt; =C2=A0storage.owner-uid: 36<br&gt;<br&gt;<br&gt; =C2=A0but this didn&#39;t affected performace in any big way<br&gt; =C2=A0should hosts to be restarted?<br&gt;<br&gt; =C2=A0_______________________________________________<br&gt; =C2=A0Users mailing list<br&gt; =C2=A0<a href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ovirt= =2Eorg</a><br&gt; =C2=A0<a href=3D"http://lists.ovirt.org/mailman/listinfo/users" target= =3D"_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br<br_______________________________________________<brUsers mailing list<br<a href=3D"mailto:Users@ovirt.org" target=3D&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt;&lt;br= <a href=3D"http://lists.ovirt.org/mailman/listinfo/users" target=3D"_blank"= >http://lists.ovirt.org/mailman/listinfo/users</a><br</div></div></blockquote></div><br></div></div --Alternative_=_Boundary_=_1399919906--