<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.E-MailFormatvorlage18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.E-MailFormatvorlage19
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=DE link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Hello,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>unfortunately I still have this problem… <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Last week I checked all the hardware components. It’s a HP DL580 Gen8 Server, 128GB RAM, 4TB storage.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>The firmware of all components is up to date.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>I ran a full check of all harddrives, CPUs etc., no problems detected.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>This night
3 VMs stopped responding again, so I had to reboot the server this morning to regain access. Some minutes ago 2 VMs stopped responding…<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>The logs just show that the VMs aren’t responding anymore, nothing else… does anybody have an idea how I can debug this issue any further?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>OS: CentOS Linux release 7.1.1503<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>>rpm -qa|grep ovirt<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-iso-uploader-3.5.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-setup-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-guest-tools-iso-3.5-7.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-log-collector-3.5.4-2.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-userportal-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-cli-3.5.0.6-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-tools-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-release35-005-1.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-lib-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-setup-plugin-ovirt-engine-common-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-host-deploy-java-1.3.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-extensions-api-impl-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-webadmin-portal-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-restapi-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-setup-base-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-backend-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-setup-plugin-websocket-proxy-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-host-deploy-1.3.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-websocket-proxy-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-dbscripts-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-jboss-as-7.1.1-1.el7.x86_64<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-sdk-python-3.5.4.0-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-setup-plugin-ovirt-engine-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-image-uploader-3.5.1-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>ovirt-engine-3.5.4.2-1.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>>rpm -qa|grep vdsm<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-python-4.16.26-0.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-jsonrpc-java-1.0.15-1.el7.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-jsonrpc-4.16.26-0.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-yajsonrpc-4.16.26-0.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-xmlrpc-4.16.26-0.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-cli-4.16.26-0.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-4.16.26-0.el7.centos.x86_64<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>vdsm-python-zombiereaper-4.16.26-0.el7.centos.noarch<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>>rpm -qa|grep kvm<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>qemu-kvm-ev-2.1.2-23.el7_1.8.1.x86_64<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>qemu-kvm-common-ev-2.1.2-23.el7_1.8.1.x86_64<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>libvirt-daemon-kvm-1.2.8-16.el7_1.3.x86_64<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>qemu-kvm-tools-ev-2.1.2-23.el7_1.8.1.x86_64<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>>uname -a <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Linux ovirt 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Any feedback is much appreciated!!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Best regards, Christian<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='mso-fareast-language:DE'>Von:</span></b><span style='mso-fareast-language:DE'> users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] <b>Im Auftrag von </b>Christian Hailer<br><b>Gesendet:</b> Samstag, 29. August 2015 22:48<br><b>An:</b> users@ovirt.org<br><b>Betreff:</b> [ovirt-users] Some VMs in status "not responding" in oVirt interface<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US>Hello,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>last Wednesday I wanted to update my oVirt 3.5 hypervisor. It is a single Centos 7 server, so I started by suspending the VMs in order to set the oVirt engine host to maintenance mode. During the process of suspending the VMs the server crashed, kernel panic…<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>After restarting the server I installed the updates via yum an restarted the server again. Afterwards, all the VMs could be started again. Some hours later my monitoring system registered some unresponsive hosts, I had a look in the oVirt interface, 3 of the VMs were in the state “not responding”, marked by a question mark. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I tried to shut down the VMs, but oVirt wasn’t able to do so. I tried to reset the status in the database with the sql statement<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>update vm_dynamic set status = 0 where vm_guid = (select vm_guid from vm_static where vm_name = 'MYVMNAME');<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>but that didn’t help, either. Only rebooting the whole hypervisor helped… afterwards everything worked again. But only for a few hours, then one of the VMs entered the “not responding” state again… again only a reboot helped. Yesterday it happened again:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>2015-08-28 17:44:22,664 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) [4ef90b12] VM DC 0f3d1f06-e516-48ce-aa6f-7273c33d3491 moved from Up --> NotResponding<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>2015-08-28 17:44:22,692 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) [4ef90b12] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC is not responding.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Does anybody know what I can do? Where should I have a look? Hints are greatly appreciated!<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thanks,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Christian<o:p></o:p></span></p></div></body></html>