<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 09 Aug 2016, at 19:14, Nicolás <<a href="mailto:nicolas@devels.es" class="">nicolas@devels.es</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
Hi,<br class="">
<br class="">
It worked (thanks Ekin, I'd probably had not turned it off and it
would indeed been restarted), restarting libvirtd and vdsmd made all
machines set their status to up. Seems the cuprit is libvirtd in
this case, as I could see some errors in the log. I'm attaching the
log FWIW.<br class=""></div></div></blockquote><div><br class=""></div><div>Hi,</div><div>I suppose you use RHEL/CentOS, a libvirt responsiveness issue was fixed in <a href="https://rhn.redhat.com/errata/RHBA-2016-1290.html" class="">https://rhn.redhat.com/errata/RHBA-2016-1290.html</a></div></div><div>Let us know if the same thing reproduces again</div><div><br class=""></div><div>Thanks,</div><div>michal</div><div><br class=""><blockquote type="cite" class=""><div class=""><div bgcolor="#FFFFFF" text="#000000" class="">
<br class="">
Thanks.<br class="">
<br class="">
<div class="moz-cite-prefix">El 07/08/16 a las 19:37, Ekin Meroğlu
escribió:<br class="">
</div>
<blockquote cite="mid:CAA0tEyOBT7KH+Xd2NMKpKY489DDbvS3wgfbXTXJi1vCCvoQNGA@mail.gmail.com" type="cite" class="">
<div dir="ltr" class="">
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Hi,</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Just a
reminder, if you have power management configured, first turn
that off for the host - when you restart vdsmd with the power
management configured, engine finds it not responding and
tries to fence (e.g. reboot) the host.</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Other
than that, restarting vdsmd has been safe in my experience...</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Regards,
</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Thu, Aug 4, 2016 at 6:10 PM, Nicolás
<span dir="ltr" class=""><<a moz-do-not-send="true" href="mailto:nicolas@devels.es" target="_blank" class="">nicolas@devels.es</a>></span>
wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><br class="">
<br class="">
El 04/08/16 a las 15:25, Arik Hadas escribió:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="">
<div class="h5">
<br class="">
----- Original Message -----<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
El 2016-08-04 08:24, Arik Hadas escribió:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
----- Original Message -----<br class="">
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<br class="">
El 04/08/16 a las 07:18, Arik Hadas escribió:<br class="">
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
----- Original Message -----<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
Hi,<br class="">
<br class="">
We're running oVirt 4.0.1 and today I found
out that one of our hosts<br class="">
has all its VMs in an unknown state. I
actually don't know how (and<br class="">
when) did this happen, but I'd like to
restore service possibly without<br class="">
turning off these machines. The host is up,
the VMs are up, 'qemu'<br class="">
process exists, no errors, it's just the VMs
running on it that have a<br class="">
'?' where status is defined.<br class="">
<br class="">
Is it safe in this case to simply modify
database and set those VM's<br class="">
status to 'up'? I remember having to do this
a time ago when we faced<br class="">
storage issues, it didn't break anything
back then. If not, is there a<br class="">
"safe" way to migrate those VMs to a
different host and restart the<br class="">
host<br class="">
that marked them as unknown?<br class="">
</blockquote>
Hi Nicolás,<br class="">
<br class="">
I assume that the host these VMs are running
on is empty in the<br class="">
webadmin,<br class="">
right? if that is the case then you've
probably hit [1]. Changing their<br class="">
status to up is not the way to go since these
VMs will not be monitored.<br class="">
</blockquote>
Hi Arik,<br class="">
<br class="">
By "empty" you mean the webadmin reports the
host being running 0 VMs?<br class="">
If so, that's not the case, actually the VM
count seems to be correct<br class="">
in<br class="">
relation to "qemu-*" processes (about 32 VMs), I
can even see the<br class="">
machines in the "Virtual machines" tab of the
host, it's just they are<br class="">
all marked with the '?' mark.<br class="">
</blockquote>
No, I meant the 'Host' column in the Virtual
Machines tab but if you<br class="">
see<br class="">
the VMs in the "Virtual machines" sub-tab of the
host then run_on_vds<br class="">
points to the right host..<br class="">
<br class="">
The host is up in the webadmin as well?<br class="">
Can you share the engine log?<br class="">
<br class="">
</blockquote>
Yes, the host is up in the webadmin, there are no
issues with it, just<br class="">
the VMs running on it have the '?' mark. I've made 3
tests:<br class="">
<br class="">
1) Restart engine: did not help<br class="">
2) Check firewall, seems to be ok.<br class="">
2) PostgreSQL: UPDATE vm_dynamic SET status = 1
WHERE status = 8; :<br class="">
After a while, I see lots of entries like this:<br class="">
<br class="">
2016-08-04 09:23:10,910 WARN<br class="">
[org.ovirt.engine.core.dal.dbb<wbr class="">roker.auditloghandling.AuditLo<wbr class="">gDirector]<br class="">
(DefaultQuartzScheduler4) [6ad135b8] Correlation ID:
null, Call Stack:<br class="">
null, Custom Event ID: -1, Message: VM xxx is not
responding.<br class="">
<br class="">
I'm attaching the engine log, but I don't know when
did this happen for<br class="">
the first time, though. If there's a manual
way/command to migrate VMs<br class="">
to a different host I'd appreciate a hint about it.<br class="">
<br class="">
Is it safe to restart vdsmd on this host?<br class="">
</blockquote>
</div>
</div>
The engine log looks fine - the VMs are reported as
not-responding for<br class="">
some reason. I would restart libvirtd and vdsmd then<br class="">
</blockquote>
<br class="">
Is restarting those two daemons safe? I mean, will that stop
all qemu-* processes, so the VMs marked as unknown will
stop?
<div class="HOEnZb">
<div class="h5"><br class="">
<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks.<br class="">
<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
Thanks.<br class="">
<br class="">
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
Yes, there is no other way to resolve it other
than changing the DB but<br class="">
the change should be to update run_on_vds
field of these VMs to the host<br class="">
you know they are running on. Their status
will then be updates in 15<br class="">
sec.<br class="">
<br class="">
[1] <a moz-do-not-send="true" href="https://bugzilla.redhat.com/show_bug.cgi?id=1354494" rel="noreferrer" target="_blank" class="">https://bugzilla.redhat.com/sh<wbr class="">ow_bug.cgi?id=1354494</a><br class="">
<br class="">
Arik.<br class="">
<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
Thanks.<br class="">
<br class="">
Nicolás<br class="">
<br class="">
______________________________<wbr class="">_________________<br class="">
Users mailing list<br class="">
<a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class="">
<a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a><br class="">
<br class="">
</blockquote>
</blockquote>
<br class="">
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br class="">
______________________________<wbr class="">_________________<br class="">
Users mailing list<br class="">
<a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class="">
<a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a><br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
<br clear="all" class="">
<div class=""><br class="">
</div>
-- <br class="">
<div class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div style="text-align:left" class="">
<table style="font-family:Ubuntu" class="">
<tbody class="">
<tr class="">
<td class=""><img moz-do-not-send="true" src="http://www.linuxera.com/files/RH_CertifiedArchitect_CMYK_small.png" class=""></td>
<td valign="top" class=""><font face="arial, helvetica,
sans-serif" class=""><strong style="font-size:small" class="">Ekin
Meroğlu</strong><i style="font-size:small" class=""> Red
Hat Certified Architect</i><font size="2" class=""> </font><br class="">
</font><font face="arial,
helvetica, sans-serif" class=""><br class="">
<font size="2" class="">linuxera
Özgür Yazılım Çözüm ve
Hizmetleri </font><br class="">
<strong style="font-size:small" class="">T</strong><font size="2" class=""> +90 (850) 22
LINUX | </font><strong style="font-size:small" class="">GSM</strong><font size="2" class=""> +90 (532) 137 77
04</font><br class="">
<font size="2" class=""><a moz-do-not-send="true" href="http://www.linuxera.com/" target="_blank" class="">www.linuxera.com</a>
| <a moz-do-not-send="true" href="mailto:bilgi@linuxera.com" target="_blank" class="">bilgi@linuxera.com</a></font></font></td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br class="">
</div>
<span id="cid:76301FB7-7C1B-4C58-8D45-6F5F4DEE8F7A@brq.redhat.com"><libvirtd.tar.gz></span>_______________________________________________<br class="">Users mailing list<br class=""><a href="mailto:Users@ovirt.org" class="">Users@ovirt.org</a><br class="">http://lists.ovirt.org/mailman/listinfo/users<br class=""></div></blockquote></div><br class=""></body></html>