<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 09 Aug 2016, at 19:14, Nicolás &lt;<a href="mailto:nicolas@devels.es" class="">nicolas@devels.es</a>&gt; wrote:</div><br class="Apple-interchange-newline"><div class="">
  
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type" class="">
  
  <div bgcolor="#FFFFFF" text="#000000" class="">
    Hi,<br class="">
    <br class="">
    It worked (thanks Ekin, I'd probably had not turned it off and it
    would indeed been restarted), restarting libvirtd and vdsmd made all
    machines set their status to up. Seems the cuprit is libvirtd in
    this case, as I could see some errors in the log. I'm attaching the
    log FWIW.<br class=""></div></div></blockquote><div><br class=""></div><div>Hi,</div><div>I suppose you use RHEL/CentOS, a libvirt responsiveness issue was fixed in&nbsp;<a href="https://rhn.redhat.com/errata/RHBA-2016-1290.html" class="">https://rhn.redhat.com/errata/RHBA-2016-1290.html</a></div></div><div>Let us know if the same thing reproduces again</div><div><br class=""></div><div>Thanks,</div><div>michal</div><div><br class=""><blockquote type="cite" class=""><div class=""><div bgcolor="#FFFFFF" text="#000000" class="">
    <br class="">
    Thanks.<br class="">
    <br class="">
    <div class="moz-cite-prefix">El 07/08/16 a las 19:37, Ekin Meroğlu
      escribió:<br class="">
    </div>
    <blockquote cite="mid:CAA0tEyOBT7KH+Xd2NMKpKY489DDbvS3wgfbXTXJi1vCCvoQNGA@mail.gmail.com" type="cite" class="">
      <div dir="ltr" class="">
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Hi,</div>
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br class="">
        </div>
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Just a
          reminder, if you have power management configured, first turn
          that off for the host - when you restart vdsmd with the power
          management configured, engine finds it not responding and
          tries to fence (e.g. reboot) the host.</div>
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br class="">
        </div>
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Other
          than that, restarting vdsmd has been safe in my experience...</div>
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br class="">
        </div>
        <div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Regards,
          &nbsp;</div>
      </div>
      <div class="gmail_extra"><br class="">
        <div class="gmail_quote">On Thu, Aug 4, 2016 at 6:10 PM, Nicolás
          <span dir="ltr" class="">&lt;<a moz-do-not-send="true" href="mailto:nicolas@devels.es" target="_blank" class="">nicolas@devels.es</a>&gt;</span>
          wrote:<br class="">
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex"><br class="">
            <br class="">
            El 04/08/16 a las 15:25, Arik Hadas escribió:<br class="">
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div class="">
                <div class="h5">
                  <br class="">
                  ----- Original Message -----<br class="">
                  <blockquote class="gmail_quote" style="margin:0 0 0
                    .8ex;border-left:1px #ccc solid;padding-left:1ex">
                    El 2016-08-04 08:24, Arik Hadas escribió:<br class="">
                    <blockquote class="gmail_quote" style="margin:0 0 0
                      .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      ----- Original Message -----<br class="">
                      <blockquote class="gmail_quote" style="margin:0 0
                        0 .8ex;border-left:1px #ccc
                        solid;padding-left:1ex">
                        <br class="">
                        El 04/08/16 a las 07:18, Arik Hadas escribió:<br class="">
                        <blockquote class="gmail_quote" style="margin:0
                          0 0 .8ex;border-left:1px #ccc
                          solid;padding-left:1ex">
                          ----- Original Message -----<br class="">
                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex">
                            Hi,<br class="">
                            <br class="">
                            We're running oVirt 4.0.1 and today I found
                            out that one of our hosts<br class="">
                            has all its VMs in an unknown state. I
                            actually don't know how (and<br class="">
                            when) did this happen, but I'd like to
                            restore service possibly without<br class="">
                            turning off these machines. The host is up,
                            the VMs are up, 'qemu'<br class="">
                            process exists, no errors, it's just the VMs
                            running on it that have a<br class="">
                            '?' where status is defined.<br class="">
                            <br class="">
                            Is it safe in this case to simply modify
                            database and set those VM's<br class="">
                            status to 'up'? I remember having to do this
                            a time ago when we faced<br class="">
                            storage issues, it didn't break anything
                            back then. If not, is there a<br class="">
                            "safe" way to migrate those VMs to a
                            different host and restart the<br class="">
                            host<br class="">
                            that marked them as unknown?<br class="">
                          </blockquote>
                          Hi Nicolás,<br class="">
                          <br class="">
                          I assume that the host these VMs are running
                          on is empty in the<br class="">
                          webadmin,<br class="">
                          right? if that is the case then you've
                          probably hit [1]. Changing their<br class="">
                          status to up is not the way to go since these
                          VMs will not be monitored.<br class="">
                        </blockquote>
                        Hi Arik,<br class="">
                        <br class="">
                        By "empty" you mean the webadmin reports the
                        host being running 0 VMs?<br class="">
                        If so, that's not the case, actually the VM
                        count seems to be correct<br class="">
                        in<br class="">
                        relation to "qemu-*" processes (about 32 VMs), I
                        can even see the<br class="">
                        machines in the "Virtual machines" tab of the
                        host, it's just they are<br class="">
                        all marked with the '?' mark.<br class="">
                      </blockquote>
                      No, I meant the 'Host' column in the Virtual
                      Machines tab but if you<br class="">
                      see<br class="">
                      the VMs in the "Virtual machines" sub-tab of the
                      host then run_on_vds<br class="">
                      points to the right host..<br class="">
                      <br class="">
                      The host is up in the webadmin as well?<br class="">
                      Can you share the engine log?<br class="">
                      <br class="">
                    </blockquote>
                    Yes, the host is up in the webadmin, there are no
                    issues with it, just<br class="">
                    the VMs running on it have the '?' mark. I've made 3
                    tests:<br class="">
                    <br class="">
                    1) Restart engine: did not help<br class="">
                    2) Check firewall, seems to be ok.<br class="">
                    2) PostgreSQL: UPDATE vm_dynamic SET status = 1
                    WHERE status = 8; :<br class="">
                    After a while, I see lots of entries like this:<br class="">
                    <br class="">
                    &nbsp; &nbsp; &nbsp; 2016-08-04 09:23:10,910 WARN<br class="">
                    [org.ovirt.engine.core.dal.dbb<wbr class="">roker.auditloghandling.AuditLo<wbr class="">gDirector]<br class="">
                    (DefaultQuartzScheduler4) [6ad135b8] Correlation ID:
                    null, Call Stack:<br class="">
                    null, Custom Event ID: -1, Message: VM xxx is not
                    responding.<br class="">
                    <br class="">
                    I'm attaching the engine log, but I don't know when
                    did this happen for<br class="">
                    the first time, though. If there's a manual
                    way/command to migrate VMs<br class="">
                    to a different host I'd appreciate a hint about it.<br class="">
                    <br class="">
                    Is it safe to restart vdsmd on this host?<br class="">
                  </blockquote>
                </div>
              </div>
              The engine log looks fine - the VMs are reported as
              not-responding for<br class="">
              some reason. I would restart libvirtd and vdsmd then<br class="">
            </blockquote>
            <br class="">
            Is restarting those two daemons safe? I mean, will that stop
            all qemu-* processes, so the VMs marked as unknown will
            stop?
            <div class="HOEnZb">
              <div class="h5"><br class="">
                <br class="">
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <blockquote class="gmail_quote" style="margin:0 0 0
                    .8ex;border-left:1px #ccc solid;padding-left:1ex">
                    Thanks.<br class="">
                    <br class="">
                    <blockquote class="gmail_quote" style="margin:0 0 0
                      .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <blockquote class="gmail_quote" style="margin:0 0
                        0 .8ex;border-left:1px #ccc
                        solid;padding-left:1ex">
                        Thanks.<br class="">
                        <br class="">
                        <blockquote class="gmail_quote" style="margin:0
                          0 0 .8ex;border-left:1px #ccc
                          solid;padding-left:1ex">
                          Yes, there is no other way to resolve it other
                          than changing the DB but<br class="">
                          the change should be to update run_on_vds
                          field of these VMs to the host<br class="">
                          you know they are running on. Their status
                          will then be updates in 15<br class="">
                          sec.<br class="">
                          <br class="">
                          [1] <a moz-do-not-send="true" href="https://bugzilla.redhat.com/show_bug.cgi?id=1354494" rel="noreferrer" target="_blank" class="">https://bugzilla.redhat.com/sh<wbr class="">ow_bug.cgi?id=1354494</a><br class="">
                          <br class="">
                          Arik.<br class="">
                          <br class="">
                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex">
                            Thanks.<br class="">
                            <br class="">
                            Nicolás<br class="">
                            <br class="">
                            ______________________________<wbr class="">_________________<br class="">
                            Users mailing list<br class="">
                            <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class="">
                            <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a><br class="">
                            <br class="">
                          </blockquote>
                        </blockquote>
                        <br class="">
                      </blockquote>
                    </blockquote>
                  </blockquote>
                </blockquote>
                <br class="">
                ______________________________<wbr class="">_________________<br class="">
                Users mailing list<br class="">
                <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class="">
                <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a><br class="">
              </div>
            </div>
          </blockquote>
        </div>
        <br class="">
        <br clear="all" class="">
        <div class=""><br class="">
        </div>
        -- <br class="">
        <div class="gmail_signature" data-smartmail="gmail_signature">
          <div dir="ltr" class="">
            <div class="">
              <div dir="ltr" class="">
                <div class="">
                  <div dir="ltr" class="">
                    <div class="">
                      <div dir="ltr" class="">
                        <div dir="ltr" class="">
                          <div dir="ltr" class="">
                            <div dir="ltr" class="">
                              <div dir="ltr" class="">
                                <div style="text-align:left" class="">
                                  <table style="font-family:Ubuntu" class="">
                                    <tbody class="">
                                      <tr class="">
                                        <td class=""><img moz-do-not-send="true" src="http://www.linuxera.com/files/RH_CertifiedArchitect_CMYK_small.png" class=""></td>
                                        <td valign="top" class=""><font face="arial, helvetica,
                                            sans-serif" class=""><strong style="font-size:small" class="">Ekin
                                              Meroğlu</strong><i style="font-size:small" class="">&nbsp;Red
                                              Hat Certified Architect</i><font size="2" class="">&nbsp;</font><br class="">
                                          </font><font face="arial,
                                            helvetica, sans-serif" class=""><br class="">
                                            <font size="2" class="">linuxera
                                              Özgür Yazılım Çözüm ve
                                              Hizmetleri&nbsp;</font><br class="">
                                            <strong style="font-size:small" class="">T</strong><font size="2" class="">&nbsp;+90 (850) 22
                                              LINUX&nbsp;|&nbsp;</font><strong style="font-size:small" class="">GSM</strong><font size="2" class="">&nbsp;+90 (532) 137 77
                                              04</font><br class="">
                                            <font size="2" class=""><a moz-do-not-send="true" href="http://www.linuxera.com/" target="_blank" class="">www.linuxera.com</a>
                                              | <a moz-do-not-send="true" href="mailto:bilgi@linuxera.com" target="_blank" class="">bilgi@linuxera.com</a></font></font></td>
                                      </tr>
                                    </tbody>
                                  </table>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br class="">
  </div>

<span id="cid:76301FB7-7C1B-4C58-8D45-6F5F4DEE8F7A@brq.redhat.com">&lt;libvirtd.tar.gz&gt;</span>_______________________________________________<br class="">Users mailing list<br class=""><a href="mailto:Users@ovirt.org" class="">Users@ovirt.org</a><br class="">http://lists.ovirt.org/mailman/listinfo/users<br class=""></div></blockquote></div><br class=""></body></html>