<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <p>Correct,<br>
      coincidentally, there no instances of systemd-udevd going haywire
      either.<br>
    </p>
    <p>Andrea<br>
    </p>
    <br>
    <div class="moz-cite-prefix">On 14/02/2017 15:20, Yaniv Kaul wrote:<br>
    </div>
    <blockquote
cite="mid:CAJgorsboDgsDRNHgyuYt_xnZcnL9nAh4x73+XMpgECarJk2gFQ@mail.gmail.com"
      type="cite">
      <div dir="ltr"><br>
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Tue, Feb 14, 2017 at 2:15 PM,
            Andrea Fagiani <span dir="ltr">&lt;<a
                moz-do-not-send="true"
                href="mailto:andrea.fagiani@immobiliare.it"
                target="_blank">andrea.fagiani@immobiliare.it</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000">
                <p>Hi,</p>
                <p>just a quick update on the issue. The high amount of
                  IOPS was indeed due to having the hosted-engine on the
                  EQL storage; we worked around the issue by moving the
                  hosted-engine to a gluster storage instead.</p>
              </div>
            </blockquote>
            <div><br>
            </div>
            <div>And now there are no high amount of IOPS on the Gluster
              storage?</div>
            <div>Y.</div>
            <div><br>
            </div>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000">
                <p>Thanks to everyone involved,<br>
                  Andrea<br>
                </p>
                <div>
                  <div class="h5"> <br>
                    <div class="m_-7339259678222976733moz-cite-prefix">On
                      13/02/2017 13:24, Nir Soffer wrote:<br>
                    </div>
                    <blockquote type="cite">
                      <div dir="ltr">
                        <div class="gmail_extra">
                          <div class="gmail_quote">On Mon, Feb 13, 2017
                            at 12:32 PM, Andrea Fagiani <span dir="ltr">&lt;<a
                                moz-do-not-send="true"
                                href="mailto:andrea.fagiani@immobiliare.it"
                                target="_blank">andrea.fagiani@immobiliare.it</a><wbr>&gt;</span>
                            wrote:<br>
                            <blockquote class="gmail_quote"
                              style="margin:0 0 0 .8ex;border-left:1px
                              #ccc solid;padding-left:1ex">
                              <div bgcolor="#FFFFFF" text="#000000">
                                <p>Hi Nir,</p>
                                <p>the amount of IOPS is pretty much
                                  constant, attaching the logs from the
                                  last hour or so.</p>
                              </div>
                            </blockquote>
                            <div><br>
                            </div>
                            <div>Andrea, can you send also logs from
                              ovirt-hosted-engine-ha?</div>
                            <div>(I'm not sure about the name of the
                              service or the location of the logs)</div>
                            <div> </div>
                            <blockquote class="gmail_quote"
                              style="margin:0 0 0 .8ex;border-left:1px
                              #ccc solid;padding-left:1ex">
                              <div bgcolor="#FFFFFF" text="#000000"><span
                                  class="m_-7339259678222976733HOEnZb"><font
                                    color="#888888">
                                    <p>Andrea<br>
                                    </p>
                                  </font></span>
                                <div>
                                  <div class="m_-7339259678222976733h5">
                                    <br>
                                    <div
                                      class="m_-7339259678222976733m_1481480387959180211moz-cite-prefix">On
                                      13/02/2017 10:13, Nir Soffer
                                      wrote:<br>
                                    </div>
                                    <blockquote type="cite">
                                      <div dir="ltr">
                                        <div>
                                          <div class="gmail_extra">
                                            <div class="gmail_quote">On
                                              Mon, Feb 13, 2017 at 11:05
                                              AM, Andrea Fagiani <span
                                                dir="ltr">&lt;<a
                                                  moz-do-not-send="true"
href="mailto:andrea.fagiani@immobiliare.it" target="_blank">andrea.fagiani@immobiliare.it</a><wbr>&gt;</span>
                                              wrote:<br>
                                              <blockquote
                                                class="gmail_quote"
                                                style="margin:0px 0px
                                                0px
                                                0.8ex;border-left:1px
                                                solid
                                                rgb(204,204,204);padding-left:1ex">Thanks
                                                for the heads up, I'll
                                                take a look at Mark's
                                                thread.<br>
                                                We are presenting 2
                                                storage LUNs to a single
                                                ovirt cluster and yes,
                                                all hosts are hosted
                                                engine hosts.<br>
                                                <br>
                                                Thanks,<br>
                                                Andrea</blockquote>
                                              <div><br>
                                              </div>
                                              <div>I think we already
                                                have hosted engine bug
                                                about doing to many
                                                calls to vdsm</div>
                                              <div>that cause reloads of
                                                lvm data; this can cause
                                                increased io.</div>
                                              <div><br>
                                              </div>
                                              Can you attach vdsm and
                                              hosted engine agent logs
                                              showing the timeframe when
                                              <div>you have lot of iops
                                                on the server?</div>
                                              <div><br>
                                              </div>
                                              <div>Nir<br>
                                              </div>
                                              <div> </div>
                                              <blockquote
                                                class="gmail_quote"
                                                style="margin:0px 0px
                                                0px
                                                0.8ex;border-left:1px
                                                solid
                                                rgb(204,204,204);padding-left:1ex">
                                                <div
                                                  class="m_-7339259678222976733m_1481480387959180211gmail-HOEnZb">
                                                  <div
                                                    class="m_-7339259678222976733m_1481480387959180211gmail-h5"><br>
                                                    <br>
                                                    <br>
                                                    On 12/02/2017 18:27,
                                                    Andrea Ghelardi
                                                    wrote:<br>
                                                    <blockquote
                                                      class="gmail_quote"
                                                      style="margin:0px
                                                      0px 0px
                                                      0.8ex;border-left:1px
                                                      solid
                                                      rgb(204,204,204);padding-left:1ex">
                                                      Andrea,<br>
                                                      I don't want to
                                                      point any fingers
                                                      but Mark Greenall
                                                      faced some issues
                                                      with Ovirt
                                                      4.06/4.1 linked to
                                                      equallogic as
                                                      well.<br>
                                                      You may want to
                                                      search posts in
                                                      this ML with
                                                      subject " Ovirt
                                                      4.0.6 guests 'Not
                                                      Responding' " to
                                                      see if you find
                                                      any similarities.<br>
                                                      How many storages
                                                      are you presenting
                                                      to how many
                                                      clusters?<br>
                                                      Also, are your
                                                      nodes all Hosted
                                                      Engine Hosts? In
                                                      my old
                                                      configuration,
                                                      normal hosts read
                                                      IOPS ranged from
                                                      50 to 500 while HE
                                                      ones peaked even
                                                      ~3000<br>
                                                      Cheers<br>
                                                      AG<br>
                                                      <br>
                                                      <br>
                                                      -----Original
                                                      Message-----<br>
                                                      From: Andrea
                                                      Fagiani [mailto:<a
moz-do-not-send="true" href="mailto:andrea.fagiani@immobiliare.it"
                                                        target="_blank">andrea.fagiani@immobil<wbr>iare.it</a>]<br>
                                                      Sent: Friday,
                                                      February 10, 2017
                                                      5:56 PM<br>
                                                      To: Andrea
                                                      Ghelardi &lt;<a
                                                        moz-do-not-send="true"
href="mailto:a.ghelardi@iontrading.com" target="_blank">a.ghelardi@iontrading.com</a>&gt;;
                                                      <a
                                                        moz-do-not-send="true"
href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
                                                      Subject: Re:
                                                      [ovirt-users] high
                                                      number of IOPS
                                                      from ovirt hosts<br>
                                                      <br>
                                                      Hi Andrea,<br>
                                                      <br>
                                                      thanks for the
                                                      insight; no I
                                                      wasn't able to
                                                      pinpoint the issue
                                                      to a single host,
                                                      the load seems to
                                                      be evenly spread.<br>
                                                      We have another
                                                      cluster running
                                                      4.0.6 on a
                                                      compellent as well
                                                      and we haven't
                                                      experienced any
                                                      similar issues.<br>
                                                      <br>
                                                      Thanks,<br>
                                                      Andrea<br>
                                                      <br>
                                                      <br>
                                                      On 10/02/2017
                                                      17:17, Andrea
                                                      Ghelardi wrote:<br>
                                                      <blockquote
                                                        class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
                                                        rgb(204,204,204);padding-left:1ex">
                                                        Hello Andrea,<br>
                                                        I saw that
                                                        happening on my
                                                        HE hosts
                                                        connected to a
                                                        COMPELLENT SC040<br>
                                                        when I was
                                                        running Ovirt
                                                        3.5<br>
                                                        <br>
                                                        All HE hosts
                                                        were hammering
                                                        the SAN with
                                                        2~4k IOPS each<br>
                                                        <br>
                                                        Upgrading to
                                                        4.0.5 solved my
                                                        case.<br>
                                                        Are you able to
                                                        check if IOPS
                                                        are generated
                                                        from a
                                                        particular host
                                                        or rather spread
                                                        among all nodes?<br>
                                                        <br>
                                                        Cheers<br>
                                                        AG<br>
                                                        <br>
                                                        <br>
                                                        -----Original
                                                        Message-----<br>
                                                        From: <a
                                                          moz-do-not-send="true"
href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a>
                                                        [mailto:<a
                                                          moz-do-not-send="true"
href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.or<wbr>g</a>]
                                                        On<br>
                                                        Behalf Of Andrea
                                                        Fagiani<br>
                                                        Sent: Friday,
                                                        February 10,
                                                        2017 12:47 PM<br>
                                                        To: <a
                                                          moz-do-not-send="true"
href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
                                                        Subject:
                                                        [ovirt-users]
                                                        high number of
                                                        IOPS from ovirt
                                                        hosts<br>
                                                        <br>
                                                        Hi,<br>
                                                        <br>
                                                        We've got a
                                                        8-host cluster
                                                        running oVirt
                                                        4.1; each host
                                                        is connected via
                                                        iscsi to a EQL
                                                        4100 storage
                                                        (2x).<br>
                                                        Most of the VMs
                                                        use barely any
                                                        IOPS, however on
                                                        our storage
                                                        controller we've
                                                        noticed a
                                                        constant steam
                                                        of reads, about
                                                        2000 IOPS; any
                                                        idea what might
                                                        cause the ovirt
                                                        hosts to hammer
                                                        on the storage
                                                        so much?<br>
                                                        <br>
                                                        Looking at
                                                        iotop, it seems
                                                        that
                                                        systemd-udevd is
                                                        being activated
                                                        every few
                                                        seconds (and its
                                                        IO% is usually
                                                        fairly high) so
                                                        I assume this
                                                        might be somehow
                                                        related.<br>
                                                        <br>
                                                        Thanks,<br>
                                                        Andrea<br>
______________________________<wbr>_________________<br>
                                                        Users mailing
                                                        list<br>
                                                        <a
                                                          moz-do-not-send="true"
href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
                                                        <a
                                                          moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
                                                          target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
                                                      </blockquote>
                                                    </blockquote>
                                                    <br>
______________________________<wbr>_________________<br>
                                                    Users mailing list<br>
                                                    <a
                                                      moz-do-not-send="true"
href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
                                                    <a
                                                      moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
                                                      target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
                                                  </div>
                                                </div>
                                              </blockquote>
                                            </div>
                                            <br>
                                          </div>
                                        </div>
                                      </div>
                                    </blockquote>
                                    <br>
                                  </div>
                                </div>
                              </div>
                            </blockquote>
                          </div>
                          <br>
                        </div>
                      </div>
                    </blockquote>
                    <br>
                  </div>
                </div>
              </div>
              <br>
              ______________________________<wbr>_________________<br>
              Users mailing list<br>
              <a moz-do-not-send="true" href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
              <a moz-do-not-send="true"
                href="http://lists.ovirt.org/mailman/listinfo/users"
                rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
              <br>
            </blockquote>
          </div>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>