<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    My apologies, I'm showing how much of a noob I am.<br>
    Ignore last direct to gluster numbers, as that wasn't really
    glusterfs.<br>
    <br>
    <br>
    [root@ovirt2 test ~]# mount -t glusterfs
    ovirt2-ks.test.j2noc.com:/gv1 /mnt/tmp/<br>
    [root@ovirt2 test ~]# time dd if=/dev/zero of=/mnt/tmp/testfile2
    bs=1M count=1000 oflag=direct<br>
    1048576000 bytes (1.0 GB) copied, 65.8596 s, 15.9 MB/s<br>
    <br>
    That's more how I expected, it is pointing to glusterfs performance.<br>
    <br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 02/11/2016 03:27 PM, Bill James
      wrote:<br>
    </div>
    <blockquote cite="mid:56BD18D7.8080909@j2.com" type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      don't know if it helps, but I ran a few more tests, all from the
      same hardware node.<br>
      <br>
      The VM:<br>
      [root@billjov1 ~]# time dd if=/dev/zero of=/root/testfile bs=1M
      count=1000 oflag=direct<br>
      1048576000 bytes (1.0 GB) copied, 62.5535 s, 16.8 MB/s<br>
      <br>
      Writing directly to gluster volume:<br>
      [root@ovirt2 test ~]# time dd if=/dev/zero
      of=/gluster-store/brick1/gv1/testfile bs=1M count=1000
      oflag=direct<br>
      1048576000 bytes (1.0 GB) copied, 9.92048 s, 106 MB/s<br>
      <br>
      <br>
      Writing to NFS volume:<br>
      [root@ovirt2 test ~]# time dd if=/dev/zero
      of=/mnt/storage/qa/testfile bs=1M count=1000 oflag=direct<br>
      1048576000 bytes (1.0 GB) copied, 10.5776 s, 99.1 MB/s<br>
      <br>
      NFS &amp; Gluster are using the same interface. Tests were not run
      at same time.<br>
      <br>
      This would suggest my problem isn't glusterfs, but the VM
      performance.<br>
      <br>
      <br>
      <br>
      <div class="moz-cite-prefix">On 02/11/2016 03:13 PM, Bill James
        wrote:<br>
      </div>
      <blockquote cite="mid:56BD1589.8090506@j2.com" type="cite">xml
        attached. <br>
        <br>
        <br>
        On 02/11/2016 12:28 PM, Nir Soffer wrote: <br>
        <blockquote type="cite">On Thu, Feb 11, 2016 at 8:27 PM, Bill
          James <a moz-do-not-send="true" class="moz-txt-link-rfc2396E"
            href="mailto:bill.james@j2.com">&lt;bill.james@j2.com&gt;</a>
          wrote: <br>
          <blockquote type="cite">thank you for the reply. <br>
            <br>
            We setup gluster using the names associated with  NIC 2 IP.
            <br>
              Brick1: ovirt1-ks.test.j2noc.com:/gluster-store/brick1/gv1
            <br>
              Brick2: ovirt2-ks.test.j2noc.com:/gluster-store/brick1/gv1
            <br>
              Brick3: ovirt3-ks.test.j2noc.com:/gluster-store/brick1/gv1
            <br>
            <br>
            That's NIC 2's IP. <br>
            Using 'iftop -i eno2 -L 5 -t' : <br>
            <br>
            dd if=/dev/zero of=/root/testfile bs=1M count=1000
            oflag=direct <br>
            1048576000 bytes (1.0 GB) copied, 68.0714 s, 15.4 MB/s <br>
          </blockquote>
          Can you share the xml of this vm? You can find it in vdsm log,
          <br>
          at the time you start the vm. <br>
          <br>
          Or you can do (on the host): <br>
          <br>
          # virsh <br>
          virsh # list <br>
          (username: vdsm@ovirt password: shibboleth) <br>
          virsh # dumpxml vm-id <br>
          <br>
          <blockquote type="cite">Peak rate
            (sent/received/total):                      281Mb 5.36Mb <br>
            282Mb <br>
            Cumulative (sent/received/total):                    1.96GB
            14.6MB <br>
            1.97GB <br>
            <br>
            gluster volume info gv1: <br>
              Options Reconfigured: <br>
              performance.write-behind-window-size: 4MB <br>
              performance.readdir-ahead: on <br>
              performance.cache-size: 1GB <br>
              performance.write-behind: off <br>
            <br>
            performance.write-behind: off didn't help. <br>
            Neither did any other changes I've tried. <br>
            <br>
            <br>
            There is no VM traffic on this VM right now except my test.
            <br>
            <br>
            <br>
            <br>
            On 02/10/2016 11:55 PM, Nir Soffer wrote: <br>
            <blockquote type="cite">On Thu, Feb 11, 2016 at 2:42 AM,
              Ravishankar N <a moz-do-not-send="true"
                class="moz-txt-link-rfc2396E"
                href="mailto:ravishankar@redhat.com">&lt;ravishankar@redhat.com&gt;</a>
              <br>
              wrote: <br>
              <blockquote type="cite">+gluster-users <br>
                <br>
                Does disabling 'performance.write-behind' give a better
                throughput? <br>
                <br>
                <br>
                <br>
                On 02/10/2016 11:06 PM, Bill James wrote: <br>
                <blockquote type="cite">I'm setting up a ovirt cluster
                  using glusterfs and noticing not stellar <br>
                  performance. <br>
                  Maybe my setup could use some adjustments? <br>
                  <br>
                  3 hardware nodes running centos7.2, glusterfs 3.7.6.1,
                  ovirt 3.6.2.6-1. <br>
                  Each node has 8 spindles configured in 1 array which
                  is split using LVM <br>
                  with one logical volume for system and one for
                  gluster. <br>
                  They each have 4 NICs, <br>
                     NIC1 = ovirtmgmt <br>
                     NIC2 = gluster  (1GbE) <br>
                </blockquote>
              </blockquote>
              How do you ensure that gluster trafic is using this nic? <br>
              <br>
              <blockquote type="cite">
                <blockquote type="cite">   NIC3 = VM traffic <br>
                </blockquote>
              </blockquote>
              How do you ensure that vm trafic is using this nic? <br>
              <br>
              <blockquote type="cite">
                <blockquote type="cite">I tried with default glusterfs
                  settings <br>
                </blockquote>
              </blockquote>
              And did you find any difference? <br>
              <br>
              <blockquote type="cite">
                <blockquote type="cite">and also with: <br>
                  performance.cache-size: 1GB <br>
                  performance.readdir-ahead: on <br>
                  performance.write-behind-window-size: 4MB <br>
                  <br>
                  [root@ovirt3 test scripts]# gluster volume info gv1 <br>
                  <br>
                  Volume Name: gv1 <br>
                  Type: Replicate <br>
                  Volume ID: 71afc35b-09d7-4384-ab22-57d032a0f1a2 <br>
                  Status: Started <br>
                  Number of Bricks: 1 x 3 = 3 <br>
                  Transport-type: tcp <br>
                  Bricks: <br>
                  Brick1:
                  ovirt1-ks.test.j2noc.com:/gluster-store/brick1/gv1 <br>
                  Brick2:
                  ovirt2-ks.test.j2noc.com:/gluster-store/brick1/gv1 <br>
                  Brick3:
                  ovirt3-ks.test.j2noc.com:/gluster-store/brick1/gv1 <br>
                  Options Reconfigured: <br>
                  performance.cache-size: 1GB <br>
                  performance.readdir-ahead: on <br>
                  performance.write-behind-window-size: 4MB <br>
                  <br>
                  <br>
                  Using simple dd test on VM in ovirt: <br>
                      dd if=/dev/zero of=/root/testfile bs=1G count=1
                  oflag=direct <br>
                </blockquote>
              </blockquote>
              block size of 1G?! <br>
              <br>
              Try 1M (our default for storage operations) <br>
              <br>
              <blockquote type="cite">
                <blockquote type="cite">        1073741824 bytes (1.1
                  GB) copied, 65.9337 s, 16.3 MB/s <br>
                  <br>
                  Another VM not in ovirt using nfs: <br>
                       dd if=/dev/zero of=/root/testfile bs=1G count=1
                  oflag=direct <br>
                          1073741824 bytes (1.1 GB) copied, 27.0079 s,
                  39.8 MB/s <br>
                  <br>
                  <br>
                  Is that expected or is there a better way to set it up
                  to get better <br>
                  performance? <br>
                </blockquote>
              </blockquote>
              Adding Niels for advice. <br>
              <br>
              <blockquote type="cite">
                <blockquote type="cite">This email, its contents and
                  .... <br>
                </blockquote>
              </blockquote>
              Please avoid this, this is a public mailing list,
              everything you write <br>
              here is public. <br>
              <br>
              Nir <br>
            </blockquote>
            I'll have to look into how to remove this sig for this
            mailing list.... <br>
            <br>
            Cloud Services for Business <a moz-do-not-send="true"
              class="moz-txt-link-abbreviated" href="http://www.j2.com">www.j2.com</a>
            <br>
            j2 | eFax | eVoice | FuseMail | Campaigner | KeepItSafe |
            Onebox <br>
            <br>
            <br>
            This email, its contents and attachments contain information
            from j2 Global, <br>
            Inc. and/or its affiliates which may be privileged,
            confidential or <br>
            otherwise protected from disclosure. The information is
            intended to be for <br>
            the addressee(s) only. If you are not an addressee, any
            disclosure, copy, <br>
            distribution, or use of the contents of this message is
            prohibited. If you <br>
            have received this email in error please notify the sender
            by reply e-mail <br>
            and delete the original message and any copies. (c) 2015 j2
            Global, Inc. All <br>
            rights reserved. eFax, eVoice, Campaigner, FuseMail,
            KeepItSafe, and Onebox <br>
            are registered trademarks of j2 Global, Inc. and its
            affiliates. <br>
          </blockquote>
        </blockquote>
        <br>
        <br>
        <fieldset class="mimeAttachmentHeader"></fieldset>
        <br>
        <pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
      </blockquote>
      <br>
      <p><a moz-do-not-send="true"
href="http://www.j2.com/?utm_source=j2global&amp;utm_medium=xsell-referral&amp;utm_campaign=employeeemail"><span
            style="color:windowtext;
            text-decoration:none"><img moz-do-not-send="true"
src="http://home.j2.com/j2_Global_Cloud_Services/j2_Global_Email_Footer.jpg"
              alt="www.j2.com" border="0" height="46" width="391"></span></a></p>
      <p><span
style="font-size:8.0pt;font-family:&quot;Arial&quot;,&quot;sans-serif&quot;;color:gray">This
          email, its contents and attachments contain information from <a
            moz-do-not-send="true"
href="http://www.j2.com/?utm_source=j2global&amp;utm_medium=xsell-referral&amp;utm_campaign=employemail">j2
            Global, Inc</a>. and/or its affiliates which may be
          privileged, confidential or otherwise protected from
          disclosure. The information is intended to be for the
          addressee(s) only. If you are not an addressee, any
          disclosure, copy, distribution, or use of the contents of this
          message is prohibited. If you have received this email in
          error please notify the sender by reply e-mail and delete the
          original message and any copies. © 2015 <a
            moz-do-not-send="true" href="http://www.j2.com/">j2 Global,
            Inc</a>. All rights reserved. <a moz-do-not-send="true"
            href="http://www.efax.com/">eFax ®</a>, <a
            moz-do-not-send="true" href="http://www.evoice.com/">eVoice
            ®</a>, <a moz-do-not-send="true"
            href="http://www.campaigner.com/">Campaigner ®</a>, <a
            moz-do-not-send="true" href="http://www.fusemail.com/">FuseMail
            ®</a>, <a moz-do-not-send="true"
            href="http://www.keepitsafe.com/">KeepItSafe ®</a> and <a
            moz-do-not-send="true" href="http://www.onebox.com/">Onebox
            ®</a> are r egistered trademarks of <a
            moz-do-not-send="true" href="http://www.j2.com/">j2 Global,
            Inc</a>. and its affiliates.</span></p>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  
<p><a href="http://www.j2.com/?utm_source=j2global&utm_medium=xsell-referral&utm_campaign=employeeemail"><span style='color:windowtext;
text-decoration:none'><img border=0 width=391 height=46
src="http://home.j2.com/j2_Global_Cloud_Services/j2_Global_Email_Footer.jpg" alt="www.j2.com"></span></a></p>

<p><span style='font-size:8.0pt;font-family:"Arial","sans-serif";
color:gray'>This email, its contents and attachments contain information from <a href="http://www.j2.com/?utm_source=j2global&utm_medium=xsell-referral&utm_campaign=employemail">j2 Global, Inc</a>. and/or its affiliates which may be privileged, confidential or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution, or use of the contents of this message is prohibited. If you have received this email in error please notify the sender by reply e-mail and delete the original message and any copies. © 2015 <a href="http://www.j2.com/">j2 Global, Inc</a>. All rights reserved. <a href="http://www.efax.com/">eFax ®</a>, <a href="http://www.evoice.com/">eVoice ®</a>, <a href="http://www.campaigner.com/">Campaigner ®</a>, <a href="http://www.fusemail.com/">FuseMail ®</a>, <a href="http://www.keepitsafe.com/">KeepItSafe ®</a> and <a href="http://www.onebox.com/">Onebox ®</a> are registered trademarks of <a href="http://www.j2.com/">j2 Global, Inc</a>. and its affiliates.</span></p></body>
</html>