<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Hi Yaniv,</div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Just a reminder, can you give us a pointer? Red Hat Support just asked us to disable PM before restarting vdsm again.</div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Thanks & Best regards,</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 22, 2016 at 10:57 PM, Ekin Meroğlu <span dir="ltr"><<a href="mailto:ekin.meroglu@linuxera.com" target="_blank">ekin.meroglu@linuxera.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Hi Yaniv,</div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"><br></div><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Sun, Aug 7, 2016 at 9:37 PM, Ekin Meroğlu <span dir="ltr"><<a href="mailto:ekin.meroglu@linuxera.com" target="_blank">ekin.meroglu@linuxera.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div style="font-family:tahoma,sans-serif;font-size:small">Hi,</div><div style="font-family:tahoma,sans-serif;font-size:small"><br></div><div style="font-family:tahoma,sans-serif;font-size:small">Just a reminder, if you have power management configured, first turn that off for the host - when you restart vdsmd with the power management configured, engine finds it not responding and tries to fence (e.g. reboot) the host.</div></div></blockquote><div><br></div></span><div>That's not true - if it's a graceful restart, it should not happen.</div></div></div></div></blockquote><div><br></div></span><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline">Can you explain this a little more? Is there a mechanism to prevent fencing on this scenario? </div></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline"><br></div></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline">In two of our customers' production systems we've experienced this exact behavior (i.e. engine fencing the host while restarting vdsm service manually) for a number of times, and we were specifically advised by Red Hat Support to turn off PM before restarting service. I'd like to to know if we have a better / easier way to restart vdsm. </div></div><div><br></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline">btw, b</div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline">oth of the environments were RHEV-H based RHEV 3.5 clusters, and both we were busy systems, so restarting vdsm service took quite a long time. I'm guessing this might be a factor.</div></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline"><br></div></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;display:inline">Regards,</div></div><div><div class="h5"><div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small"></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="m_7539410844558501315HOEnZb"><font color="#888888"></font></span><div><div class="m_7539410844558501315h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div style="font-family:tahoma,sans-serif;font-size:small"><br></div><div style="font-family:tahoma,sans-serif;font-size:small">Other than that, restarting vdsmd has been safe in my experience...</div><div style="font-family:tahoma,sans-serif;font-size:small"><br></div><div style="font-family:tahoma,sans-serif;font-size:small">Regards, </div></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Thu, Aug 4, 2016 at 6:10 PM, Nicolás <span dir="ltr"><<a href="mailto:nicolas@devels.es" target="_blank">nicolas@devels.es</a>></span> wrote:<br></span><div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
El 04/08/16 a las 15:25, Arik Hadas escribió:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
El 2016-08-04 08:24, Arik Hadas escribió:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
El 04/08/16 a las 07:18, Arik Hadas escribió:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
We're running oVirt 4.0.1 and today I found out that one of our hosts<br>
has all its VMs in an unknown state. I actually don't know how (and<br>
when) did this happen, but I'd like to restore service possibly without<br>
turning off these machines. The host is up, the VMs are up, 'qemu'<br>
process exists, no errors, it's just the VMs running on it that have a<br>
'?' where status is defined.<br>
<br>
Is it safe in this case to simply modify database and set those VM's<br>
status to 'up'? I remember having to do this a time ago when we faced<br>
storage issues, it didn't break anything back then. If not, is there a<br>
"safe" way to migrate those VMs to a different host and restart the<br>
host<br>
that marked them as unknown?<br>
</blockquote>
Hi Nicolás,<br>
<br>
I assume that the host these VMs are running on is empty in the<br>
webadmin,<br>
right? if that is the case then you've probably hit [1]. Changing their<br>
status to up is not the way to go since these VMs will not be monitored.<br>
</blockquote>
Hi Arik,<br>
<br>
By "empty" you mean the webadmin reports the host being running 0 VMs?<br>
If so, that's not the case, actually the VM count seems to be correct<br>
in<br>
relation to "qemu-*" processes (about 32 VMs), I can even see the<br>
machines in the "Virtual machines" tab of the host, it's just they are<br>
all marked with the '?' mark.<br>
</blockquote>
No, I meant the 'Host' column in the Virtual Machines tab but if you<br>
see<br>
the VMs in the "Virtual machines" sub-tab of the host then run_on_vds<br>
points to the right host..<br>
<br>
The host is up in the webadmin as well?<br>
Can you share the engine log?<br>
<br>
</blockquote>
Yes, the host is up in the webadmin, there are no issues with it, just<br>
the VMs running on it have the '?' mark. I've made 3 tests:<br>
<br>
1) Restart engine: did not help<br>
2) Check firewall, seems to be ok.<br>
2) PostgreSQL: UPDATE vm_dynamic SET status = 1 WHERE status = 8; :<br>
After a while, I see lots of entries like this:<br>
<br>
2016-08-04 09:23:10,910 WARN<br>
[org.ovirt.engine.core.dal.dbb<wbr>roker.auditloghandling.AuditLo<wbr>gDirector]<br>
(DefaultQuartzScheduler4) [6ad135b8] Correlation ID: null, Call Stack:<br>
null, Custom Event ID: -1, Message: VM xxx is not responding.<br>
<br>
I'm attaching the engine log, but I don't know when did this happen for<br>
the first time, though. If there's a manual way/command to migrate VMs<br>
to a different host I'd appreciate a hint about it.<br>
<br>
Is it safe to restart vdsmd on this host?<br>
</blockquote></div></div>
The engine log looks fine - the VMs are reported as not-responding for<br>
some reason. I would restart libvirtd and vdsmd then<br>
</blockquote>
<br>
Is restarting those two daemons safe? I mean, will that stop all qemu-* processes, so the VMs marked as unknown will stop?<div><div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Yes, there is no other way to resolve it other than changing the DB but<br>
the change should be to update run_on_vds field of these VMs to the host<br>
you know they are running on. Their status will then be updates in 15<br>
sec.<br>
<br>
[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1354494" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/sh<wbr>ow_bug.cgi?id=1354494</a><br>
<br>
Arik.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks.<br>
<br>
Nicolás<br>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br>
</blockquote></blockquote>
<br>
</blockquote></blockquote></blockquote></blockquote>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</div></div></blockquote></div></div></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="text-align:left"><table style="font-family:Ubuntu"><tbody><tr><td><img src="http://www.linuxera.com/files/RH_CertifiedArchitect_CMYK_small.png"></td><td valign="top"><font face="arial, helvetica, sans-serif"><strong style="font-size:small">Ekin Meroğlu</strong><i style="font-size:small"> Red Hat Certified Architect</i><font size="2"> </font><br></font><font face="arial, helvetica, sans-serif"><br><font size="2">linuxera Özgür Yazılım Çözüm ve Hizmetleri </font><br><strong style="font-size:small">T</strong><font size="2"> +90 (850) 22 LINUX | </font><strong style="font-size:small">GSM</strong><font size="2"> <a href="tel:%2B90%20%28532%29%20137%2077%2004" value="+905321377704" target="_blank">+90 (532) 137 77 04</a></font><br><font size="2"><a href="http://www.linuxera.com" target="_blank">www.linuxera.com</a> | <a href="mailto:bilgi@linuxera.com" target="_blank">bilgi@linuxera.com</a></font></font></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div></div></div>
</font></span></div>
<br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></blockquote></div></div></div><br></div></div>
</blockquote></div></div></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br><div class="m_7539410844558501315gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="text-align:left"><table style="font-family:Ubuntu"><tbody><tr><td><img src="http://www.linuxera.com/files/RH_CertifiedArchitect_CMYK_small.png"></td><td valign="top"><font face="arial, helvetica, sans-serif"><strong style="font-size:small">Ekin Meroğlu</strong><i style="font-size:small"> Red Hat Certified Architect</i><font size="2"> </font><br></font><font face="arial, helvetica, sans-serif"><br><font size="2">linuxera Özgür Yazılım Çözüm ve Hizmetleri </font><br><strong style="font-size:small">T</strong><font size="2"> +90 (850) 22 LINUX | </font><strong style="font-size:small">GSM</strong><font size="2"> <a href="tel:%2B90%20%28532%29%20137%2077%2004" value="+905321377704" target="_blank">+90 (532) 137 77 04</a></font><br><font size="2"><a href="http://www.linuxera.com" target="_blank">www.linuxera.com</a> | <a href="mailto:bilgi@linuxera.com" target="_blank">bilgi@linuxera.com</a></font></font></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="text-align:left"><table style="font-family:Ubuntu"><tbody><tr><td><img src="http://www.linuxera.com/files/RH_CertifiedArchitect_CMYK_small.png"></td><td valign="top"><font face="arial, helvetica, sans-serif"><strong style="font-size:small">Ekin Meroğlu</strong><i style="font-size:small"> Red Hat Certified Architect</i><font size="2"> </font><br></font><font face="arial, helvetica, sans-serif"><br><font size="2">linuxera Özgür Yazılım Çözüm ve Hizmetleri </font><br><strong style="font-size:small">T</strong><font size="2"> +90 (850) 22 LINUX | </font><strong style="font-size:small">GSM</strong><font size="2"> +90 (532) 137 77 04</font><br><font size="2"><a href="http://www.linuxera.com" target="_blank">www.linuxera.com</a> | <a href="mailto:bilgi@linuxera.com" target="_blank">bilgi@linuxera.com</a></font></font></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>