<div dir="ltr">Hi Sven,<div>libgfapi is not fully operational yet.</div><div>There's some additional work which just got merged[1] in order to enable it.</div><div>Hopefully it'll be included in one of the next releases.</div><div><br></div><div>Doron</div><div><br></div><div>[1] <a href="https://gerrit.ovirt.org/#/c/78938/">https://gerrit.ovirt.org/#/c/78938/</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 9 July 2017 at 14:34, Sven Achtelik <span dir="ltr"><<a href="mailto:Sven.Achtelik@eps.aero" target="_blank">Sven.Achtelik@eps.aero</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="white" lang="DE" link="blue" vlink="purple"><div class="m_5227293379229981623WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Hi All, <u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">it’s the same for me. I’ve update all my hosts to the latest release and thought it would now use libgfapi since <a href="https://bugzilla.redhat.com/1022961" target="_blank"><span style="color:#1f497d;text-decoration:none">BZ 1022961</span></a> is listed in the release notes under enhancements. Are there any steps that need to be taken after upgrading for this to work ? <u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Thank you, <u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Sven <u></u><u></u></span></p><p class="MsoNormal"><a name="m_5227293379229981623__MailEndCompose"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></a></p><div><div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">Von:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> <a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a> [mailto:<a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.<wbr>org</a>] <b>Im Auftrag von </b>Mahdi Adnan<br><b>Gesendet:</b> Samstag, 8. Juli 2017 09:35<br><b>An:</b> Ralf Schenk <<a href="mailto:rs@databay.de" target="_blank">rs@databay.de</a>>; <a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>; <a href="mailto:ykaul@redhat.com" target="_blank">ykaul@redhat.com</a><br><b>Betreff:</b> Re: [ovirt-users] Very poor GlusterFS performance<u></u><u></u></span></p></div></div><div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><div id="m_5227293379229981623divtagdefaultwrapper"><p><span style="font-family:"Calibri",sans-serif">So ovirt access gluster vai FUSE ? i thought its using libgfapi.<u></u><u></u></span></p><p><span style="font-family:"Calibri",sans-serif">When can we expect it to work with libgfapi ? <u></u><u></u></span></p><p><span style="font-family:"Calibri",sans-serif">and what about the changelog of 4.1.3 ?<u></u><u></u></span></p><div><p class="MsoNormal"><span style="font-family:"Calibri",sans-serif">BZ 1022961 Gluster: running a VM from a gluster domain should use gluster URI instead of a fuse mount"<u></u><u></u></span></p></div><p><span style="font-family:"Calibri",sans-serif"><u></u> <u></u></span></p><div id="m_5227293379229981623Signature"><p class="MsoNormal"><span style="font-family:"Calibri",sans-serif"><u></u> <u></u></span></p><div><p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Calibri",sans-serif">-- <br><br>Respectfully<b><br>Mahdi A. Mahdi</b><u></u><u></u></span></p></div></div></div><div class="MsoNormal" align="center" style="text-align:center"><hr size="4" width="98%" align="center"></div><div id="m_5227293379229981623divRplyFwdMsg"><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> <a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a> <<a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a>> on behalf of Ralf Schenk <<a href="mailto:rs@databay.de" target="_blank">rs@databay.de</a>><br><b>Sent:</b> Monday, June 19, 2017 7:32:45 PM<br><b>To:</b> <a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br><b>Subject:</b> Re: [ovirt-users] Very poor GlusterFS performance</span> <u></u><u></u></p><div><p class="MsoNormal"> <u></u><u></u></p></div></div><div><p><span style="font-family:"Helvetica",sans-serif">Hello,</span><u></u><u></u></p><p><span style="font-family:"Helvetica",sans-serif">Gluster-Performance is bad. Thats why I asked for native qemu-libgfapi access for Ovirt-VM's to gluster volumes which I thought to be possible since 3.6.x. Documentation is misleading and still in 4.1.2 Ovirt is using fuse to mount gluster-based VM-Disks.</span><u></u><u></u></p><p>Bye<u></u><u></u></p><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal">Am 19.06.2017 um 17:23 schrieb Darrell Budic:<u></u><u></u></p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><p class="MsoNormal">Chris- <u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">You probably need to head over to <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a> for help with performance issues.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">That said, what kind of performance are you getting, via some form or testing like bonnie++ or even dd runs? Raw bricks vs gluster performance is useful to determine what kind of performance you’re actually getting.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Beyond that, I’d recommend dropping the arbiter bricks and re-adding them as full replicas, they can’t serve distributed data in this configuration and may be slowing things down on you. If you’ve got a storage network setup, make sure it’s using the largest MTU it can, and consider adding/testing these settings that I use on my main storage volume:<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><div><p class="MsoNormal"><a href="http://performance.io" target="_blank">performance.io</a>-thread-count: 32<u></u><u></u></p></div><div><p class="MsoNormal">client.event-threads: 8<u></u><u></u></p></div><div><p class="MsoNormal">server.event-threads: 3<u></u><u></u></p></div><div><p class="MsoNormal">performance.stat-prefetch: on<u></u><u></u></p></div></div><div><p class="MsoNormal"><br><br><u></u><u></u></p></div><div><p class="MsoNormal">Good luck,<u></u><u></u></p></div><div><p class="MsoNormal"><br><br><u></u><u></u></p></div><div><p class="MsoNormal"> -Darrell<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p><div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal">On Jun 19, 2017, at 9:46 AM, Chris Boot <<a href="mailto:bootc@bootc.net" target="_blank">bootc@bootc.net</a>> wrote:<u></u><u></u></p></div><p class="MsoNormal"><u></u> <u></u></p><div><div><p class="MsoNormal">Hi folks,<br><br>I have 3x servers in a "hyper-converged" oVirt 4.1.2 + GlusterFS 3.10<br>configuration. My VMs run off a replica 3 arbiter 1 volume comprised of<br>6 bricks, which themselves live on two SSDs in each of the servers (one<br>brick per SSD). The bricks are XFS on LVM thin volumes straight onto the<br>SSDs. Connectivity is 10G Ethernet.<br><br>Performance within the VMs is pretty terrible. I experience very low<br>throughput and random IO is really bad: it feels like a latency issue.<br>On my oVirt nodes the SSDs are not generally very busy. The 10G network<br>seems to run without errors (iperf3 gives bandwidth measurements of >=<br>9.20 Gbits/sec between the three servers).<br><br>To put this into perspective: I was getting better behaviour from NFS4<br>on a gigabit connection than I am with GlusterFS on 10G: that doesn't<br>feel right at all.<br><br>My volume configuration looks like this:<br><br>Volume Name: vmssd<br>Type: Distributed-Replicate<br>Volume ID: d5a5ddd1-a140-4e0d-b514-<wbr>701cfe464853<br>Status: Started<br>Snapshot Count: 0<br>Number of Bricks: 2 x (2 + 1) = 6<br>Transport-type: tcp<br>Bricks:<br>Brick1: ovirt3:/gluster/ssd0_vmssd/<wbr>brick<br>Brick2: ovirt1:/gluster/ssd0_vmssd/<wbr>brick<br>Brick3: ovirt2:/gluster/ssd0_vmssd/<wbr>brick (arbiter)<br>Brick4: ovirt3:/gluster/ssd1_vmssd/<wbr>brick<br>Brick5: ovirt1:/gluster/ssd1_vmssd/<wbr>brick<br>Brick6: ovirt2:/gluster/ssd1_vmssd/<wbr>brick (arbiter)<br>Options Reconfigured:<br>nfs.disable: on<br>transport.address-family: inet6<br>performance.quick-read: off<br>performance.read-ahead: off<br><a href="http://performance.io" target="_blank">performance.io</a>-cache: off<br>performance.stat-prefetch: off<br>performance.low-prio-threads: 32<br>network.remote-dio: off<br>cluster.eager-lock: enable<br>cluster.quorum-type: auto<br>cluster.server-quorum-type: server<br>cluster.data-self-heal-<wbr>algorithm: full<br>cluster.locking-scheme: granular<br>cluster.shd-max-threads: 8<br>cluster.shd-wait-qlength: 10000<br>features.shard: on<br>user.cifs: off<br>storage.owner-uid: 36<br>storage.owner-gid: 36<br>features.shard-block-size: 128MB<br>performance.strict-o-direct: on<br>network.ping-timeout: 30<br>cluster.granular-entry-heal: enable<br><br>I would really appreciate some guidance on this to try to improve things<br>because at this rate I will need to reconsider using GlusterFS altogether.<br><br>Cheers,<br>Chris<br><br>-- <br>Chris Boot<br><a href="mailto:bootc@bootc.net" target="_blank">bootc@bootc.net</a><br>______________________________<wbr>_________________<br>Users mailing list<br><a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br><a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><u></u><u></u></p></div></div></blockquote></div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal"><br><br><br><u></u><u></u></p><pre>______________________________<wbr>_________________<u></u><u></u></pre><pre>Users mailing list<u></u><u></u></pre><pre><a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><u></u><u></u></pre><pre><a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><u></u><u></u></pre></blockquote><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal">-- <u></u><u></u></p><table class="m_5227293379229981623MsoNormalTable" border="0" cellspacing="0" cellpadding="0"><tbody><tr><td colspan="3" style="padding:0cm 0cm 0cm 0cm"><p class="MsoNormal"><img border="0" width="151" height="30" id="m_5227293379229981623_x0000_i1026" src="cid:image001.gif@01D2F8B7.C9A43050"><u></u><u></u></p></td></tr><tr><td valign="top" style="padding:0cm 0cm 0cm 0cm"><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif"><br><b>Ralf Schenk</b><br>fon <a href="tel:+49%202405%20408370" value="+492405408370" target="_blank">+49 (0) 24 05 / 40 83 70</a><br>fax <a href="tel:+49%202405%204083759" value="+4924054083759" target="_blank">+49 (0) 24 05 / 40 83 759</a><br>mail <a href="mailto:rs@databay.de" target="_blank"><b><span style="color:red">rs@databay.de</span></b></a></span><u></u><u></u></p></td><td width="33" style="width:22.5pt;padding:0cm 0cm 0cm 0cm"><p class="MsoNormal"> <u></u><u></u></p></td><td valign="top" style="padding:0cm 0cm 0cm 0cm"><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif"><br><b>Databay AG</b><br>Jens-Otto-Krag-Straße 11<br>D-52146 Würselen<br><a href="http://www.databay.de" target="_blank"><b><span style="color:red">www.databay.de</span></b></a> </span><u></u><u></u></p></td></tr><tr><td colspan="3" valign="top" style="padding:0cm 0cm 0cm 0cm"><p class="MsoNormal"><span style="font-size:7.5pt;font-family:"Verdana",sans-serif"><br>Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br>Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br>Aufsichtsratsvorsitzender: Wilhelm Dohmen </span><u></u><u></u></p></td></tr></tbody></table><div class="MsoNormal" align="center" style="text-align:center"><hr size="1" width="100%" noshade style="color:black" align="center"></div></div></div></div></div></div></div><br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>