6:13 p.m.
--_000_f25a1076729a4b24a9712b1c5c7f8a9fteemlmbx11phqtargetcom_
Content-Type: text/plain; charset="iso-2022-jp"
Content-Transfer-Encoding: quoted-printable
Hey Ovirt Users and Team,
I have a host that I am unable to recover post a network outage. The host =
is stuck in unresponsive mode, even though the host is on the network, able=
to SSH and seems to be healthy. I=1B$B!G=1B(Bve tried several things to r=
ecover the host in Ovirt, but have had no success so far. I=1B$B!G=1B(Bd l=
ike to reach out to the community before blowing away and rebuilding the ho=
st.
Environment: I have an Ovengine server with about 26 Datacenters, with 2 to=
3 hosts per Datacenter. My Ovengine server is hosted centrally, with my h=
osts being bare-metal and distributed throughout my environment. Ovengin=
e is version 4.0.6.
What I=1B$B!G=1B(Bve tried: put into maintenance mode, rebooted the host. =
Confirmed host was rebooted and tried to active, goes back to unresponsive.=
Attempted a reinstall, which fails.
Checking from the host perspective, I can see the following problems:
[boxname~]# systemctl status vdsmd
=1B$B!|=1B(B vdsmd.service - Virtual Desktop Server Manager
Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor p=
reset: enabled)
Active: inactive (dead)
Jul 14 12:34:28 boxname systemd[1]: Dependency failed for Virtual Desktop S=
erver Manager.
Jul 14 12:34:28 boxname systemd[1]: Job vdsmd.service/start failed with res=
ult 'dependency'.
Going a bit deeper, the results of journalctl -xe:
[root@boxname ~]# journalctl -xe
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun shutting down.
Jul 18 09:07:31 boxname systemd[1]: Stopped Virtualization daemon.
-- Subject: Unit libvirtd.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has finished shutting down.
Jul 18 09:07:31 boxname systemd[1]: Reloading.
Jul 18 09:07:31 boxname systemd[1]: Binding to IPv6 address not available s=
ince kernel does not support IPv6.
Jul 18 09:07:31 boxname systemd[1]: [/usr/lib/systemd/system/rpcbind.socket=
:6] Failed to parse address value, ignoring: [::
Jul 18 09:07:31 boxname systemd[1]: Started Auxiliary vdsm service for runn=
ing helper functions as root.
-- Subject: Unit supervdsmd.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit supervdsmd.service has finished starting up.
--
-- The start-up result is done.
Jul 18 09:07:31 boxname systemd[1]: Starting Auxiliary vdsm service for run=
ning helper functions as root...
-- Subject: Unit supervdsmd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit supervdsmd.service has begun starting up.
Jul 18 09:07:31 boxname systemd[1]: Starting Virtualization daemon...
-- Subject: Unit libvirtd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun starting up.
Jul 18 09:07:32 boxname systemd[1]: Started Virtualization daemon.
-- Subject: Unit libvirtd.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has finished starting up.
--
-- The start-up result is done.
Jul 18 09:07:32 boxname systemd[1]: Starting Virtual Desktop Server Manager=
network restoration...
-- Subject: Unit vdsm-network.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit vdsm-network.service has begun starting up.
lines 2751-2797/2797 (END)
Does the community have suggestions on what can be done next to recover thi=
s host within Ovirt? I can provide additional log dumps as needed, please =
inform with what you need to assist further.
Thank you,
Tony
--_000_f25a1076729a4b24a9712b1c5c7f8a9fteemlmbx11phqtargetcom_
Content-Type: text/html; charset="iso-2022-jp"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml"
xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word"
=
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml"
xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html;
charset=3Diso-2022-=
jp">
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1"
vlink=3D"#954F72">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hey Ovirt Users and Team,<o:p
</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">I have a host that I am unable to recover post a
net=
work outage. The host is stuck in unresponsive mode, even though the =
host is on the network, able to SSH and seems to be healthy. I=1B$B!G=
=1B(Bve tried several things to recover the host in Ovirt,
but have had no success so far. I=1B$B!G=1B(Bd like to reach out to =
the community before blowing away and rebuilding the host.<o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><b>Environment</b>: I have an Ovengine
server with a=
bout 26 Datacenters, with 2 to 3 hosts per Datacenter. My Ovengine se=
rver is hosted centrally, with my hosts being bare-metal and distributed th=
roughout my environment. Ovengine is
version 4.0.6. <o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><b>What I=1B$B!G=1B(Bve tried: </b>put
into maintena=
nce mode, rebooted the host. Confirmed host was rebooted and tried to=
active, goes back to unresponsive. Attempted a reinstall, whic=
h fails.
<o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><b>Checking from the host perspective, I can
see the=
following problems:
<o:p></o:p></b></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">[boxname~]# systemctl status vdsmd<o:p</o:p></p>
<p
class=3D"MsoNormal">=1B$B!|=1B(B vdsmd.service - Virtual Desktop Server =
Manager<o:p</o:p></p>
<p
class=3D"MsoNormal"> Loaded: loaded
(/usr/lib/systemd/system=
/vdsmd.service; enabled; vendor preset: enabled)<o:p</o:p></p>
<p
class=3D"MsoNormal"> Active: inactive (dead)<o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Jul 14 12:34:28 boxname systemd[1]: Dependency
faile=
d for Virtual Desktop Server Manager.<o:p</o:p></p>
<p
class=3D"MsoNormal">Jul 14 12:34:28 boxname systemd[1]: Job vdsmd.servic=
e/start failed with result 'dependency'.<o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><b>Going a bit deeper, the results of
journalctl =
211;xe: <o:p></o:p></b></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">[root@boxname ~]# journalctl -xe<o:p</o:p></p>
<p class=3D"MsoNormal">--
Defined-By: systemd<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit libvirtd.service has begun shutting down.<o:=
p</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]: Stopped Virtuali=
zation daemon.<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Subject: Unit libvirtd.service has finished shutt=
ing down<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Defined-By: systemd<o:p</o:p></p>
<p class=3D"MsoNormal">--
Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit libvirtd.service has finished shutting down.=
<o:p</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]:
Reloading.<o:p><=
/o:p></p>
<p class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]: Binding to IPv6
=
address not available since kernel does not support IPv6.<o:p</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]: [/usr/lib/system=
d/system/rpcbind.socket:6] Failed to parse address value, ignoring: [::<o:p=
</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]: Started Auxiliar=
y vdsm service for running helper functions as root.<o:p</o:p></p>
<p class=3D"MsoNormal">--
Subject: Unit supervdsmd.service has finished sta=
rt-up<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Defined-By: systemd<o:p</o:p></p>
<p class=3D"MsoNormal">--
Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit supervdsmd.service has finished starting up.=
<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
The start-up result is done.<o:p</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]: Starting Auxilia=
ry vdsm service for running helper functions as root...<o:p</o:p></p>
<p class=3D"MsoNormal">--
Subject: Unit supervdsmd.service has begun start-=
up<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Defined-By: systemd<o:p</o:p></p>
<p class=3D"MsoNormal">--
Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit supervdsmd.service has begun starting up.<o:=
p</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:31 boxname systemd[1]: Starting Virtual=
ization daemon...<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Subject: Unit libvirtd.service has begun start-up=
<o:p</o:p></p>
<p class=3D"MsoNormal">--
Defined-By: systemd<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit libvirtd.service has begun starting up.<o:p>=
</o:p></p>
<p class=3D"MsoNormal">Jul 18 09:07:32 boxname systemd[1]: Started
Virtuali=
zation daemon.<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Subject: Unit libvirtd.service has finished start=
-up<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Defined-By: systemd<o:p</o:p></p>
<p class=3D"MsoNormal">--
Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit libvirtd.service has finished starting up.<o=
:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
The start-up result is done.<o:p</o:p></p>
<p
class=3D"MsoNormal">Jul 18 09:07:32 boxname systemd[1]: Starting Virtual=
Desktop Server Manager network restoration...<o:p</o:p></p>
<p class=3D"MsoNormal">--
Subject: Unit vdsm-network.service has begun star=
t-up<o:p</o:p></p>
<p
class=3D"MsoNormal">-- Defined-By: systemd<o:p</o:p></p>
<p class=3D"MsoNormal">--
Support: http://lists.freedesktop.org/mailman/lis=
tinfo/systemd-devel<o:p</o:p></p>
<p
class=3D"MsoNormal">--<o:p</o:p></p>
<p class=3D"MsoNormal">--
Unit vdsm-network.service has begun starting up.<=
o:p</o:p></p>
<p
class=3D"MsoNormal">lines 2751-2797/2797 (END)<o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Does the community have suggestions on what can be
d=
one next to recover this host within Ovirt? I can provide additional =
log dumps as needed, please inform with what you need to assist further.<o:=
p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Thank you,<o:p</o:p></p>
<p
class=3D"MsoNormal">Tony<o:p</o:p></p>
<p
class=3D"MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>
--_000_f25a1076729a4b24a9712b1c5c7f8a9fteemlmbx11phqtargetcom_--