9:38 a.m.
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2SOCRATESasl_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hi All,
after a successful setup of hosted-engine on the first node I'm having trou=
ble completing it on an additional node. The Setup fails with:
---------------------------------
[ INFO ] Waiting for the host to become operational in the engine. This ma=
y take several minutes...
[ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] The VDSM Host is now operational
[ INFO ] Enabling and starting HA services
[ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl'
fa=
iled to execute
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answer=
s/answers-20150426080028.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
---------------------------------
After that the node is added to the cluster and is operational from the GUI=
, but the hosted engine broker and agent fail to start with error messages=
:
----------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring A=
gent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20=
min ago
Process: 5373 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-agent start (=
code=3Dexited, status=3D1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hd=
lr =3D FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi=
le "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: St=
reamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi=
le "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: st=
ream =3D open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IO=
Error: [Errno 6] No such device or address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [F=
AILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.servi=
ce: control process exited, code=3Dexited status=3D1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir=
t Hosted Engine High Availability Monitoring Agent.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.=
service entered failed state.
-------------------------------------
And
-------------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communicati=
ons Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled=
)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21=
min ago
Process: 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-broker start =
(code=3Dexited, status=3D1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: h=
dlr =3D FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F=
ile "/usr/lib64/python2.7/logging/__init__.py", line ...it__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: S=
treamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F=
ile "/usr/lib64/python2.7/logging/__init__.py", line ...open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: s=
tream =3D open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: I=
OError: [Errno 6] No such device or address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [=
FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.serv=
ice: control process exited, code=3Dexited status=3D1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir=
t Hosted Engine High Availability Communications Broker.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker=
.service entered failed state.
----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or ip=
tables. How can I find out which version of ovirt I'm running exactly ? I'=
ve had a lock at the logs and read through old bug reports.
Thank you,
Sven
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2SOCRATESasl_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml"
xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word"
=
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml"
xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type
content=
=3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator
content=3D"Micros=
oft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DDE
link=3D"#0563C1" v=
link=3D"#954F72"><div class=3DWordSection1><p
class=3DMsoNormal><span lang=
=3DEN-US>Hi All,<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-=
US><o:p> </o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>afte=
r a successful setup of hosted-engine on the first node I’m having tr=
ouble completing it on an additional node. The Setup fails with:<o:p></o:p>=
</span></p><p class=3DMsoNormal><span
lang=3DEN-US>------------------------=
---------<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>[ IN=
FO ] Waiting for the host to become operational in the engine. This m=
ay take several minutes...<o:p></o:p></span></p><p
class=3DMsoNormal><span =
lang=3DEN-US>[ INFO ] Still waiting for VDSM host to become operation=
al...<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>[ INFO&n=
bsp; ] The VDSM Host is now operational<o:p></o:p></span></p><p
class=3DMso=
Normal><span lang=3DEN-US>[ INFO ] Enabling and starting HA
services<=
o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>[ ERROR ] Fail=
ed to execute stage 'Closing up': Command '/bin/systemctl' failed to
execut=
e<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>[ INFO =
] Stage: Clean up<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DE=
N-US>[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-se=
tup/answers/answers-20150426080028.conf'<o:p></o:p></span></p><p
class=3DMs=
oNormal><span lang=3DEN-US>[ INFO ] Stage:
Pre-termination<o:p></o:p>=
</span></p><p class=3DMsoNormal><span lang=3DEN-US>[
INFO ] Stage: Te=
rmination<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>----=
-----------------------------<o:p></o:p></span></p><p
class=3DMsoNormal><sp=
an lang=3DEN-US>After that the node is added to the cluster and is operatio=
nal from the GUI, but the hosted engine broker and agent fail to star=
t with error messages: <o:p></o:p></span></p><p
class=3DMsoNormal><span lan=
g=3DEN-US>----------------------------------<o:p></o:p></span></p><p
class=
=3DMsoNormal><span lang=3DEN-US>[root@ovirt-node2 ~]# systemctl status ovir=
t-ha-agent.service -l<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=
=3DEN-US>ovirt-ha-agent.service - oVirt Hosted Engine High Availability Mon=
itoring Agent<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>=
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service=
; enabled)<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>&nb=
sp; Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 =
CDT; 20min ago<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US=
Process: 5373
ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-agent s=
tart (code=3Dexited,
status=3D1/FAILURE)<o:p></o:p></span></p><p class=3DMs=
oNormal><span
lang=3DEN-US><o:p> </o:p></span></p><p
class=3DMsoNormal=
<span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovir=
t-ha-agent[5373]: hdlr =3D FileHandler(filename,
mode)<o:p></o:p></span></p=
<p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28
ovirt-node2.mgmt.a=
sl.local systemd-ovirt-ha-agent[5373]: File
"/usr/lib64/python2.7/logg=
ing/__init__.py", line 902, in
__init__<o:p></o:p></span></p><p class=
=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local =
systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open())<o:=
p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00:28 =
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/li=
b64/python2.7/logging/__init__.py", line 925, in
_open<o:p></o:p></spa=
n></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28
ovirt-node2.m=
gmt.asl.local systemd-ovirt-ha-agent[5373]: stream =3D open(self.baseFilena=
me, self.mode)<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US=
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: I=
OError: [Errno 6] No such device or address:
'/dev/stdout'<o:p></o:p></span=
</p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26
08:00:28 ovirt-node2.mg=
mt.asl.local systemd-ovirt-ha-agent[5373]:
[FAILED]<o:p></o:p></span></p><p=
class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.=
local systemd[1]: ovirt-ha-agent.service: control process exited, code=3Dex=
ited status=3D1<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-U=
S>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oV=
irt Hosted Engine High Availability Monitoring
Agent.<o:p></o:p></span></p>=
<p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.as=
l.local systemd[1]: Unit ovirt-ha-agent.service entered failed state.<o:p><=
/o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>-------------------=
------------------<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DE=
N-US>And <o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>----=
---------------------------------<o:p></o:p></span></p><p
class=3DMsoNormal=
<span lang=3DEN-US>[root@ovirt-node2 ~]# systemctl status
ovirt-ha-broker<=
o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-US>ovirt-ha-broke=
r.service - oVirt Hosted Engine High Availability Communications Broker<o:p=
</o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US> Load=
ed: loaded
(/usr/lib/systemd/system/ovirt-ha-broker.service; enabled)<o:p><=
/o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US> Active=
: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago<o=
:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US> Process:=
5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-broker start (code=3Dex=
ited, status=3D1/FAILURE)<o:p></o:p></span></p><p
class=3DMsoNormal><span l=
ang=3DEN-US><o:p> </o:p></span></p><p
class=3DMsoNormal><span lang=3DE=
N-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[535=
9]: hdlr =3D FileHandler(filename,
mode)<o:p></o:p></span></p><p class=3DMs=
oNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local syste=
md-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.=
py", line ...it__<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=
=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker=
[5359]: StreamHandler.__init__(self,
self._open())<o:p></o:p></span></p><p =
class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.l=
ocal systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging=
/__init__.py", line ...open<o:p></o:p></span></p><p
class=3DMsoNormal>=
<span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt=
-ha-broker[5359]: stream =3D open(self.baseFilename,
self.mode)<o:p></o:p><=
/span></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28
ovirt-nod=
e2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such=
device or address:
'/dev/stdout'<o:p></o:p></span></p><p
class=3DMsoNormal=
<span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovir=
t-ha-broker[5359]:
[FAILED]<o:p></o:p></span></p><p
class=3DMsoNormal><span=
lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-=
ha-broker.service: control process exited, code=3Dexited status=3D1<o:p></o=
:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26
08:00:28 ovirt=
-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High =
Availability Communications Broker.<o:p></o:p></span></p><p
class=3DMsoNorm=
al><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]=
: Unit ovirt-ha-broker.service entered failed
state.<o:p></o:p></span></p><=
p class=3DMsoNormal><span lang=3DEN-US>------------------------------------=
----<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US><o:p>&nbs=
p;</o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US>The system is a =
CentOS 7 Setup with SeLinux switched off, no firewall or iptables. Ho=
w can I find out which version of ovirt I’m running exactly ? I’=
;ve had a lock at the logs and read through old bug
reports.<o:p></o:p></sp=
an></p><p class=3DMsoNormal><span
lang=3DEN-US><o:p> </o:p></span></p>=
<p class=3DMsoNormal><span lang=3DEN-US>Thank you,
<o:p></o:p></span></p><p=
class=3DMsoNormal><span
lang=3DEN-US><o:p> </o:p></span></p><p class=
=3DMsoNormal><span lang=3DEN-US>Sven
<o:p></o:p></span></p></div></body></h=
tml>=
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2SOCRATESasl_--
9:59 a.m.
This is a multi-part message in MIME format.
--------------070001000400050301000408
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
On 04/26/2015 05:38 PM, Sven Achtelik wrote:
Hi All,
after a successful setup of hosted-engine on the first node Im having
trouble completing it on an additional node. The Setup fails with:
---------------------------------
[ INFO ] Waiting for the host to become operational in the engine.
This may take several minutes...
[ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] The VDSM Host is now operational
[ INFO ] Enabling and starting HA services
[ ERROR ] Failed to execute stage 'Closing up': Command
'/bin/systemctl' failed to execute
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
---------------------------------
After that the node is added to the cluster and is operational from
the GUI, but the hosted engine broker and agent fail to start with
error messages:
----------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
ovirt-ha-agent.service - oVirt Hosted Engine High Availability
Monitoring Agent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service;
enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28
CDT; 20min ago
Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent
start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: File
"/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: File
"/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or
address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: [FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-agent.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
oVirt Hosted Engine High Availability Monitoring Agent.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-agent.service entered failed state.
-------------------------------------
And
-------------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
ovirt-ha-broker.service - oVirt Hosted Engine High Availability
Communications Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28
CDT; 21min ago
Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker
start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: File
"/usr/lib64/python2.7/logging/__init__.py", line ...it__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: File
"/usr/lib64/python2.7/logging/__init__.py", line ...open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or
address: '/dev/stdout'
Didi any clue?
the log says it runs as root so I canrule that out
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]: [FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-broker.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
oVirt Hosted Engine High Availability Communications Broker.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-broker.service entered failed state.
----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall
or iptables. How can I find out which version of ovirt Im running
exactly ? Ive had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--------------070001000400050301000408
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 04/26/2015 05:38 PM, Sven Achtelik
wrote:<br>
</div>
<blockquote
cite="mid:BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2@SOCRATES.asl.local"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi
All,<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">after a
successful setup
of hosted-engine on the first node Im having trouble
completing it on an additional node. The Setup fails
with:<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">---------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Waiting for
the host to become operational in the engine. This may take
several minutes...<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Still waiting
for VDSM host to become
operational...<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
The VDSM Host
is now operational<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Enabling and
starting HA services<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ ERROR ]
Failed to
execute stage 'Closing up': Command '/bin/systemctl' failed
to execute<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Stage: Clean
up<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Generating
answer file
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Stage:
Pre-termination<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[ INFO ]
Stage:
Termination<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">---------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">After that
the node is
added to the cluster and is operational from the GUI, but
the hosted engine broker and agent fail to start with error
messages: <o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">----------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">[root@ovirt-node2 ~]#
systemctl status ovirt-ha-agent.service
-l<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">ovirt-ha-agent.service -
oVirt Hosted Engine High Availability Monitoring
Agent<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Loaded:
loaded
(/usr/lib/systemd/system/ovirt-ha-agent.service;
enabled)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Active:
failed
(Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min
ago<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Process:
5373
ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
(code=exited, status=1/FAILURE)<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
hdlr = FileHandler(filename,
mode)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
File "/usr/lib64/python2.7/logging/__init__.py", line 902,
in __init__<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
StreamHandler.__init__(self,
self._open())<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
File "/usr/lib64/python2.7/logging/__init__.py", line 925,
in _open<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
stream = open(self.baseFilename,
self.mode)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
IOError: [Errno 6] No such device or address:
'/dev/stdout'<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
[FAILED]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-agent.service: control process exited, code=exited
status=1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt
Hosted Engine High Availability Monitoring
Agent.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-agent.service entered failed
state.<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">-------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">And
<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">-------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">[root@ovirt-node2 ~]#
systemctl status
ovirt-ha-broker<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">ovirt-ha-broker.service
- oVirt Hosted Engine High Availability Communications
Broker<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Loaded:
loaded
(/usr/lib/systemd/system/ovirt-ha-broker.service;
enabled)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Active:
failed
(Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min
ago<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Process:
5359
ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
(code=exited, status=1/FAILURE)<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
hdlr = FileHandler(filename,
mode)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
File "/usr/lib64/python2.7/logging/__init__.py", line
...it__<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
StreamHandler.__init__(self,
self._open())<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
File "/usr/lib64/python2.7/logging/__init__.py", line
...open<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
stream = open(self.baseFilename,
self.mode)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
IOError: [Errno 6] No such device or address:
'/dev/stdout'</span></p>
</div>
</blockquote>
<br>
Didi any clue? <br>
the log says it runs as root so I canrule that out<br>
<blockquote
cite="mid:BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2@SOCRATES.asl.local"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span
lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
[FAILED]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-broker.service: control process exited, code=exited
status=1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt
Hosted Engine High Availability Communications
Broker.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Apr 26
08:00:28
ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-broker.service entered failed
state.<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US">----------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The system
is a CentOS 7
Setup with SeLinux switched off, no firewall or iptables.
How can I find out which version of ovirt Im running
exactly ? Ive had a lock at the logs and read through old
bug reports.<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US"><o:p> </o:p></span></p>
</div>
</blockquote>
<br>
the rpm version of ovirt* will be enough I guess<br>
<br>
<blockquote
cite="mid:BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2@SOCRATES.asl.local"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Thank you,
<o:p></o:p></span></p>
<p class="MsoNormal"><span
lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Sven
<o:p></o:p></span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
</body>
</html>
--------------070001000400050301000408--
10:57 a.m.
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB7SOCRATESasl_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-=
3.5-pre
ovirt-host-deploy.noarch 1.3.1-1.el7 =
@ovirt-3.5
ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovir=
t-3.5
ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-=
pre
ovirt-release35.noarch 003-1 =
@/ovirt-release35
Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag vo=
n Roy Golan
Gesendet: Sonntag, 26. April 2015 16:59
An: users(a)ovirt.org; Yedidyah Bar David
Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote:
Hi All,
after a successful setup of hosted-engine on the first node I'm having trou=
ble completing it on an additional node. The Setup fails with:
---------------------------------
[ INFO ] Waiting for the host to become operational in the engine. This ma=
y take several minutes...
[ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] The VDSM Host is now operational
[ INFO ] Enabling and starting HA services
[ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl'
fa=
iled to execute
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answer=
s/answers-20150426080028.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
---------------------------------
After that the node is added to the cluster and is operational from the GUI=
, but the hosted engine broker and agent fail to start with error messages=
:
----------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring A=
gent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20=
min ago
Process: 5373 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-agent start (=
code=3Dexited, status=3D1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hd=
lr =3D FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi=
le "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: St=
reamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi=
le "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: st=
ream =3D open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IO=
Error: [Errno 6] No such device or address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [F=
AILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.servi=
ce: control process exited, code=3Dexited status=3D1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir=
t Hosted Engine High Availability Monitoring Agent.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.=
service entered failed state.
-------------------------------------
And
-------------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communicati=
ons Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled=
)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21=
min ago
Process: 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-broker start =
(code=3Dexited, status=3D1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: h=
dlr =3D FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F=
ile "/usr/lib64/python2.7/logging/__init__.py", line ...it__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: S=
treamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F=
ile "/usr/lib64/python2.7/logging/__init__.py", line ...open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: s=
tream =3D open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: I=
OError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue?
the log says it runs as root so I canrule that out
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [=
FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.serv=
ice: control process exited, code=3Dexited status=3D1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir=
t Hosted Engine High Availability Communications Broker.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker=
.service entered failed state.
----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or ip=
tables. How can I find out which version of ovirt I'm running exactly ? I'=
ve had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB7SOCRATESasl_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml"
xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word"
=
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml"
xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type
content=
=3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator
content=3D"Micros=
oft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Vorformatiert Zchn";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.E-MailFormatvorlage17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.HTMLVorformatiertZchn
{mso-style-name:"HTML Vorformatiert Zchn";
mso-style-priority:99;
mso-style-link:"HTML Vorformatiert";
font-family:"Consolas",serif;
color:black;
mso-fareast-language:EN-US;}
span.E-MailFormatvorlage20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body bgcolor=3Dwhite
lang=3DDE li=
nk=3D"#0563C1" vlink=3D"#954F72"><div
class=3DWordSection1><p class=3DMsoNo=
rmal><span lang=3DEN-US style=3D'color:#1F497D'>On the node that fails to
s=
tart the ha-broker and ha-agent I’m using:
<o:p></o:p></span></p><p c=
lass=3DMsoNormal><span lang=3DEN-US
style=3D'color:#1F497D'><o:p> </o:=
p></span></p><p class=3DMsoNormal><span lang=3DEN-US
style=3D'color:#1F497D=
'>ovirt-engine-sdk-python.noarch
=
3.5.2.1-1.el7.centos @ovir=
t-3.5-pre<o:p></o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US styl=
e=3D'color:#1F497D'>ovirt-host-deploy.noarch
=
&n=
bsp;
1.3.1-1.el7  =
;
&=
nbsp;@ovirt-3.5<o:p></o:p></span></p><p
class=3DMsoNormal><span lang=3DEN-U=
S style=3D'color:#1F497D'>ovirt-hosted-engine-ha.noarch
&=
nbsp; &nbs=
p; 1.2.5-1.el7.centos
@ovirt-3.5<o:p></o:p></span><=
/p><p class=3DMsoNormal><span lang=3DEN-US
style=3D'color:#1F497D'>ovirt-ho=
sted-engine-setup.noarch
1.2.3-1.=
el7.centos
@ovirt-3.5-pre<o:p></o:p></span></=
p><p class=3DMsoNormal><span lang=3DEN-US
style=3D'color:#1F497D'>ovirt-rel=
ease35.noarch
&=
nbsp;
&nb=
sp;
003-1 =
&n=
bsp; @/ovirt-release35<o:p></o:p></span>=
</p><p class=3DMsoNormal><span lang=3DEN-US
style=3D'color:#1F497D'><o:p>&n=
bsp;</o:p></span></p><p class=3DMsoNormal><span
lang=3DEN-US><o:p> </o=
:p></span></p><div><div style=3D'border:none;border-top:solid
#E1E1E1 1.0pt=
;padding:3.0pt 0cm 0cm 0cm'><p class=3DMsoNormal><b><span
style=3D'color:wi=
ndowtext;mso-fareast-language:DE'>Von:</span></b><span
style=3D'color:windo=
wtext;mso-fareast-language:DE'> users-bounces(a)ovirt.org [mailto:users-bounc=
es(a)ovirt.org] <b>Im Auftrag von </b>Roy
Golan<br><b>Gesendet:</b> Sonntag, =
26. April 2015 16:59<br><b>An:</b> users(a)ovirt.org; Yedidyah Bar
David<br><=
b>Betreff:</b> Re: [ovirt-users] Hosted Engine-Setup issue additional host<=
o:p></o:p></span></p></div></div><p
class=3DMsoNormal><o:p> </o:p></p>=
<div><p class=3DMsoNormal>On 04/26/2015 05:38 PM, Sven Achtelik
wrote:<span=
style=3D'font-size:12.0pt;mso-fareast-language:DE'><o:p></o:p></span></p><=
/div><blockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'><p
class=3D=
MsoNormal><span lang=3DEN-US>Hi
All,</span><o:p></o:p></p><p class=3DMsoNor=
mal><span
lang=3DEN-US> </span><o:p></o:p></p><p
class=3DMsoNormal><sp=
an lang=3DEN-US>after a successful setup of hosted-engine on the first node=
I’m having trouble completing it on an additional node. The Setup fa=
ils with:</span><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>----=
-----------------------------</span><o:p></o:p></p><p
class=3DMsoNormal><sp=
an lang=3DEN-US>[ INFO ] Waiting for the host to become operational i=
n the engine. This may take several
minutes...</span><o:p></o:p></p><p clas=
s=3DMsoNormal><span lang=3DEN-US>[ INFO ] Still waiting for VDSM host=
to become operational...</span><o:p></o:p></p><p
class=3DMsoNormal><span l=
ang=3DEN-US>[ INFO ] The VDSM Host is now
operational</span><o:p></o:=
p></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO ]
Enabling and s=
tarting HA services</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3D=
EN-US>[ ERROR ] Failed to execute stage 'Closing up': Command
'/bin/systemc=
tl' failed to execute</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=
=3DEN-US>[ INFO ] Stage: Clean
up</span><o:p></o:p></p><p class=3DMso=
Normal><span lang=3DEN-US>[ INFO ] Generating answer file
'/var/lib/o=
virt-hosted-engine-setup/answers/answers-20150426080028.conf'</span><o:p></=
o:p></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO ]
Stage: Pre-t=
ermination</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-US>[ I=
NFO ] Stage: Termination</span><o:p></o:p></p><p
class=3DMsoNormal><s=
pan
lang=3DEN-US>---------------------------------</span><o:p></o:p></p><p
=
class=3DMsoNormal><span lang=3DEN-US>After that the node is added to the cl=
uster and is operational from the GUI, but the hosted engine broker a=
nd agent fail to start with error messages:
</span><o:p></o:p></p><p class=
=3DMsoNormal><span
lang=3DEN-US>----------------------------------</span><o=
:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>[root@ovirt-node2 ~]# =
systemctl status ovirt-ha-agent.service
-l</span><o:p></o:p></p><p class=3D=
MsoNormal><span lang=3DEN-US>ovirt-ha-agent.service - oVirt Hosted Engine H=
igh Availability Monitoring Agent</span><o:p></o:p></p><p
class=3DMsoNormal=
<span lang=3DEN-US> Loaded: loaded
(/usr/lib/systemd/system/ov=
irt-ha-agent.service;
enabled)</span><o:p></o:p></p><p class=3DMsoNormal><s=
pan lang=3DEN-US> Active: failed (Result: exit-code) since Sun =
2015-04-26 08:00:28 CDT; 20min ago</span><o:p></o:p></p><p
class=3DMsoNorma=
l><span lang=3DEN-US> Process: 5373 ExecStart=3D/usr/lib/systemd/syst=
emd-ovirt-ha-agent start (code=3Dexited,
status=3D1/FAILURE)</span><o:p></o=
:p></p><p class=3DMsoNormal><span
lang=3DEN-US> </span><o:p></o:p></p>=
<p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.as=
l.local systemd-ovirt-ha-agent[5373]: hdlr =3D FileHandler(filename, mode)<=
/span><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00:2=
8 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/=
lib64/python2.7/logging/__init__.py", line 902, in __init__</span><o:p=
</o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00:28 ovirt-no=
de2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(sel=
f, self._open())</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-=
US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:=
File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _o=
pen</span><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:=
00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream =3D o=
pen(self.baseFilename, self.mode)</span><o:p></o:p></p><p
class=3DMsoNormal=
<span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovir=
t-ha-agent[5373]: IOError: [Errno 6] No such device or address:
'/dev/stdou=
t'</span><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:0=
0:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED]</spa=
n><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00:28 ov=
irt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control proces=
s exited, code=3Dexited status=3D1</span><o:p></o:p></p><p
class=3DMsoNorma=
l><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:=
Failed to start oVirt Hosted Engine High Availability Monitoring Agent.</s=
pan><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00:28 =
ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered =
failed state.</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-US>=
-------------------------------------</span><o:p></o:p></p><p
class=3DMsoNo=
rmal><span lang=3DEN-US>And </span><o:p></o:p></p><p
class=3DMsoNormal><spa=
n
lang=3DEN-US>-------------------------------------</span><o:p></o:p></p><=
p class=3DMsoNormal><span lang=3DEN-US>[root@ovirt-node2 ~]# systemctl stat=
us ovirt-ha-broker</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DE=
N-US>ovirt-ha-broker.service - oVirt Hosted Engine High Availability Commun=
ications Broker</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-U=
S> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.serv=
ice; enabled)</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-US>=
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:=
28 CDT; 21min ago</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN=
-US> Process: 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-brok=
er start (code=3Dexited,
status=3D1/FAILURE)</span><o:p></o:p></p><p class=
=3DMsoNormal><span
lang=3DEN-US> </span><o:p></o:p></p><p
class=3DMsoN=
ormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd=
-ovirt-ha-broker[5359]: hdlr =3D FileHandler(filename,
mode)</span><o:p></o=
:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28
ovirt-node2.=
mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2=
.7/logging/__init__.py", line
...it__</span><o:p></o:p></p><p class=3D=
MsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local sys=
temd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open())</spa=
n><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00:28 ov=
irt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib=
64/python2.7/logging/__init__.py", line
...open</span><o:p></o:p></p><=
p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl=
.local systemd-ovirt-ha-broker[5359]: stream =3D open(self.baseFilename, se=
lf.mode)</span><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 2=
6 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOErro=
r: [Errno 6] No such device or address:
'/dev/stdout'</span><o:p></o:p></p>=
</blockquote><p class=3DMsoNormal><span
style=3D'font-size:12.0pt;font-fami=
ly:"Times New Roman",serif;mso-fareast-language:DE'><br>Didi any
clue? <br>=
the log says it runs as root so I canrule that
out<br><br><o:p></o:p></span=
</p><blockquote
style=3D'margin-top:5.0pt;margin-bottom:5.0pt'><p class=3D=
MsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
sys=
temd-ovirt-ha-broker[5359]: [FAILED]</span><o:p></o:p></p><p
class=3DMsoNor=
mal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1=
]: ovirt-ha-broker.service: control process exited, code=3Dexited status=3D=
1</span><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>Apr 26 08:00=
:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Eng=
ine High Availability Communications
Broker.</span><o:p></o:p></p><p class=
=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local =
systemd[1]: Unit ovirt-ha-broker.service entered failed
state.</span><o:p><=
/o:p></p><p class=3DMsoNormal><span
lang=3DEN-US>--------------------------=
--------------</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-US=
</span><o:p></o:p></p><p
class=3DMsoNormal><span lang=3DEN-US>The sy=
stem is a CentOS 7 Setup with
SeLinux switched off, no firewall or iptables=
. How can I find out which version of ovirt I’m running exactly=
? I’ve had a lock at the logs and read through old bug reports.</spa=
n><o:p></o:p></p><p class=3DMsoNormal><span
lang=3DEN-US> </span><o:p>=
</o:p></p></blockquote><p class=3DMsoNormal><span
style=3D'font-size:12.0pt=
;font-family:"Times New
Roman",serif;mso-fareast-language:DE'><br>the rpm v=
ersion of ovirt* will be enough I
guess<br><br><br><o:p></o:p></span></p><b=
lockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'><p
class=3DMsoNorm=
al><span lang=3DEN-US>Thank you,
</span><o:p></o:p></p><p class=3DMsoNormal=
<span
lang=3DEN-US> </span><o:p></o:p></p><p
class=3DMsoNormal><span =
lang=3DEN-US>Sven
</span><o:p></o:p></p><p class=3DMsoNormal><span
style=3D=
'font-size:12.0pt;font-family:"Times New
Roman",serif;mso-fareast-language:=
DE'><br><br><br><o:p></o:p></span></p><pre>________________________________=
_______________<o:p></o:p></pre><pre>Users mailing
list<o:p></o:p></pre><pr=
e><a
href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre><p=
re><a
href=3D"http://lists.ovirt.org/mailman/listinfo/users">http:...
virt.org/mailman/listinfo/users</a><o:p></o:p></pre&...
class=
=3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times New
Roman",=
serif;mso-fareast-language:DE'><o:p> </o:p></span></p></div></body></h=
tml>=
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB7SOCRATESasl_--
1:56 a.m.
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org, "Yedidyah Bar
David" <didi(a)redhat.com>
Sent: Sunday, April 26, 2015 6:57:06 PM
Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
@ovirt-3.5-pre
ovirt-host-deploy.noarch 1.3.1-1.el7
@ovirt-3.5
ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
@ovirt-3.5
ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
@ovirt-3.5-pre
ovirt-release35.noarch 003-1
@/ovirt-release35
Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von
Roy Golan
Gesendet: Sonntag, 26. April 2015 16:59
An: users(a)ovirt.org; Yedidyah Bar David
Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote:
Hi All,
after a successful setup of hosted-engine on the first node I'm having
trouble completing it on an additional node. The Setup fails with:
---------------------------------
[ INFO ] Waiting for the host to become operational in the engine. This may
take several minutes...
[ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] The VDSM Host is now operational
[ INFO ] Enabling and starting HA services
[ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl'
failed to execute
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
---------------------------------
After that the node is added to the cluster and is operational from the GUI,
but the hosted engine broker and agent fail to start with error messages:
----------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring
Agent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
20min ago
Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
(code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr
= FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
"/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
StreamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
"/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
stream = open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
IOError: [Errno 6] No such device or address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
[FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-agent.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt
Hosted Engine High Availability Monitoring Agent.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-agent.service entered failed state.
-------------------------------------
And
-------------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
ovirt-ha-broker.service - oVirt Hosted Engine High Availability
Communications Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
21min ago
Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
(code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
hdlr = FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
File "/usr/lib64/python2.7/logging/__init__.py", line ...it__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
StreamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
File "/usr/lib64/python2.7/logging/__init__.py", line ...open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
stream = open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue?
the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout
echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]:
[FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-broker.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt
Hosted Engine High Availability Communications Broker.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-broker.service entered failed state.
----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or
iptables. How can I find out which version of ovirt I'm running exactly ?
I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
Thanks,
--
Didi
2:34 a.m.
Hi Did,
results are
---------------------------
[root@ovirt-node2 ~]# ls -l /dev/stdout
lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1
[root@ovirt-node2 ~]# echo test > /dev/stdout
test
---------------------------
Looks like everything is working fine.
Sven
-----Ursprüngliche Nachricht-----
Von: Yedidyah Bar David [mailto:didi@redhat.com]
Gesendet: Montag, 27. April 2015 08:57
An: Sven Achtelik
Cc: Roy Golan; users(a)ovirt.org
Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org, "Yedidyah Bar
David" <didi(a)redhat.com>
Sent: Sunday, April 26, 2015 6:57:06 PM
Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
@ovirt-3.5-pre
ovirt-host-deploy.noarch 1.3.1-1.el7
@ovirt-3.5
ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
@ovirt-3.5
ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
@ovirt-3.5-pre
ovirt-release35.noarch 003-1
@/ovirt-release35
Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im
Auftrag von Roy Golan
Gesendet: Sonntag, 26. April 2015 16:59
An: users(a)ovirt.org; Yedidyah Bar David
Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote:
Hi All,
after a successful setup of hosted-engine on the first node I'm having
trouble completing it on an additional node. The Setup fails with:
---------------------------------
[ INFO ] Waiting for the host to become operational in the engine.
This may take several minutes...
[ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] The VDSM Host is now operational [ INFO ] Enabling and
starting HA services [ ERROR ] Failed to execute stage 'Closing up':
Command '/bin/systemctl'
failed to execute
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
---------------------------------
After that the node is added to the cluster and is operational from
the GUI, but the hosted engine broker and agent fail to start with error messages:
----------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
ovirt-ha-agent.service - oVirt Hosted Engine High Availability
Monitoring Agent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
20min ago
Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
(code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr
26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
"/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26
08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
IOError: [Errno 6] No such device or address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
[FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-agent.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
oVirt Hosted Engine High Availability Monitoring Agent.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-agent.service entered failed state.
-------------------------------------
And
-------------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
ovirt-ha-broker.service - oVirt Hosted Engine High Availability
Communications Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
21min ago
Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
(code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
hdlr = FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26
08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26
08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue?
the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout
echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[5359]:
[FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-broker.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
oVirt Hosted Engine High Availability Communications Broker.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-broker.service entered failed state.
----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall
or iptables. How can I find out which version of ovirt I'm running exactly ?
I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
Thanks,
--
Didi
2:45 a.m.
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
To: "Yedidyah Bar David" <didi(a)redhat.com>
Cc: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org
Sent: Monday, April 27, 2015 10:34:13 AM
Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Hi Did,
results are
---------------------------
[root@ovirt-node2 ~]# ls -l /dev/stdout
lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1
[root@ovirt-node2 ~]# echo test > /dev/stdout
test
---------------------------
Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
Sven
-----Ursprüngliche Nachricht-----
Von: Yedidyah Bar David [mailto:didi@redhat.com]
Gesendet: Montag, 27. April 2015 08:57
An: Sven Achtelik
Cc: Roy Golan; users(a)ovirt.org
Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
> From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org, "Yedidyah
Bar
> David" <didi(a)redhat.com>
> Sent: Sunday, April 26, 2015 6:57:06 PM
> Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
>
> On the node that fails to start the ha-broker and ha-agent I'm using:
>
> ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
> @ovirt-3.5-pre
> ovirt-host-deploy.noarch 1.3.1-1.el7
> @ovirt-3.5
> ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
> @ovirt-3.5
> ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
> @ovirt-3.5-pre
> ovirt-release35.noarch 003-1
> @/ovirt-release35
>
>
> Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im
> Auftrag von Roy Golan
> Gesendet: Sonntag, 26. April 2015 16:59
> An: users(a)ovirt.org; Yedidyah Bar David
> Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
>
> On 04/26/2015 05:38 PM, Sven Achtelik wrote:
> Hi All,
>
> after a successful setup of hosted-engine on the first node I'm having
> trouble completing it on an additional node. The Setup fails with:
> ---------------------------------
> [ INFO ] Waiting for the host to become operational in the engine.
> This may take several minutes...
> [ INFO ] Still waiting for VDSM host to become operational...
> [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and
> starting HA services [ ERROR ] Failed to execute stage 'Closing up':
> Command '/bin/systemctl'
> failed to execute
> [ INFO ] Stage: Clean up
> [ INFO ] Generating answer file
> '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
> [ INFO ] Stage: Pre-termination
> [ INFO ] Stage: Termination
> ---------------------------------
> After that the node is added to the cluster and is operational from
> the GUI, but the hosted engine broker and agent fail to start with error
> messages:
> ----------------------------------
> [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
> ovirt-ha-agent.service - oVirt Hosted Engine High Availability
> Monitoring Agent
> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
> Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> 20min ago
> Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
> (code=exited, status=1/FAILURE)
>
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr
> 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
> "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26
> 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> IOError: [Errno 6] No such device or address: '/dev/stdout'
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> [FAILED]
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> ovirt-ha-agent.service: control process exited, code=exited status=1
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
> oVirt Hosted Engine High Availability Monitoring Agent.
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> ovirt-ha-agent.service entered failed state.
> -------------------------------------
> And
> -------------------------------------
> [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
> ovirt-ha-broker.service - oVirt Hosted Engine High Availability
> Communications Broker
> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
> enabled)
> Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> 21min ago
> Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
> (code=exited, status=1/FAILURE)
>
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> hdlr = FileHandler(filename, mode)
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26
> 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26
> 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> IOError: [Errno 6] No such device or address: '/dev/stdout'
>
> Didi any clue?
> the log says it runs as root so I canrule that out
>
That's weird. Please check/post:
ls -l /dev/stdout
echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> [FAILED]
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> ovirt-ha-broker.service: control process exited, code=exited status=1
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
> oVirt Hosted Engine High Availability Communications Broker.
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> ovirt-ha-broker.service entered failed state.
> ----------------------------------------
>
> The system is a CentOS 7 Setup with SeLinux switched off, no firewall
> or iptables. How can I find out which version of ovirt I'm running exactly
> ?
> I've had a lock at the logs and read through old bug reports.
>
>
> the rpm version of ovirt* will be enough I guess
>
>
> Thank you,
>
> Sven
>
>
>
>
> _______________________________________________
>
> Users mailing list
>
> Users@ovirt.org<mailto:Users@ovirt.org>
>
> http://lists.ovirt.org/mailman/listinfo/users
>
>
Thanks,
--
Didi
Best,
--
Didi
2:53 a.m.
Yes,
-----------------------
[root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl start
ovirt-ha-agent.service
Job for ovirt-ha-broker.service failed. See 'systemctl status
ovirt-ha-broker.service' and 'journalctl -xn' for details.
[root@ovirt-node2 ~]# journalctl -xn
-- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27 02:49:33 CDT. --
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: File
"/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: stream =
open(self.baseFilename, self.mode)
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: IOError: [Errno
6] No such device or address: '/dev/stdout'
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: [FAILED]
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control
process exited, code=exited status=1
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine
High Availability Communications Broker.
-- Subject: Unit ovirt-ha-broker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit ovirt-ha-broker.service has failed.
--
-- The result is failed.
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service
entered failed state.
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm
ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to broker, the
number of errors has exceeded the limit (
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR failed to retrieve
Hosted Engine HA info
Traceback (most recent call last):
File
"/usr/share/vdsm/API.py", line 1703, in _getHaInfo
stats =
instance.get_all_stats()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line
97, in get_all_stats
with broker.connection():
File
"/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 99, in connection
self.connect()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 78, in connect
raise
BrokerConnectionError(error_msg)
BrokerConnectionError: Failed to
connect to broker, the number of errors has exceeded the limit (5)
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not found: Requested
metadata element is not present
--------------------------------
-----Ursprüngliche Nachricht-----
Von: Yedidyah Bar David [mailto:didi@redhat.com]
Gesendet: Montag, 27. April 2015 09:46
An: Sven Achtelik; Martin Sivak
Cc: Roy Golan; users(a)ovirt.org
Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
To: "Yedidyah Bar David" <didi(a)redhat.com>
Cc: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org
Sent: Monday, April 27, 2015 10:34:13 AM
Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional
host
Hi Did,
results are
---------------------------
[root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr
26 09:14 /dev/stdout -> /proc/self/fd/1
[root@ovirt-node2 ~]# echo test > /dev/stdout test
---------------------------
Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
Sven
-----Ursprüngliche Nachricht-----
Von: Yedidyah Bar David [mailto:didi@redhat.com]
Gesendet: Montag, 27. April 2015 08:57
An: Sven Achtelik
Cc: Roy Golan; users(a)ovirt.org
Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional
host
----- Original Message -----
> From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org, "Yedidyah
Bar
> David" <didi(a)redhat.com>
> Sent: Sunday, April 26, 2015 6:57:06 PM
> Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
>
> On the node that fails to start the ha-broker and ha-agent I'm using:
>
> ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
> @ovirt-3.5-pre
> ovirt-host-deploy.noarch 1.3.1-1.el7
> @ovirt-3.5
> ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
> @ovirt-3.5
> ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
> @ovirt-3.5-pre
> ovirt-release35.noarch 003-1
> @/ovirt-release35
>
>
> Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im
> Auftrag von Roy Golan
> Gesendet: Sonntag, 26. April 2015 16:59
> An: users(a)ovirt.org; Yedidyah Bar David
> Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
>
> On 04/26/2015 05:38 PM, Sven Achtelik wrote:
> Hi All,
>
> after a successful setup of hosted-engine on the first node I'm
> having trouble completing it on an additional node. The Setup fails with:
> ---------------------------------
> [ INFO ] Waiting for the host to become operational in the engine.
> This may take several minutes...
> [ INFO ] Still waiting for VDSM host to become operational...
> [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and
> starting HA services [ ERROR ] Failed to execute stage 'Closing up':
> Command '/bin/systemctl'
> failed to execute
> [ INFO ] Stage: Clean up
> [ INFO ] Generating answer file
> '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
> [ INFO ] Stage: Pre-termination
> [ INFO ] Stage: Termination
> ---------------------------------
> After that the node is added to the cluster and is operational from
> the GUI, but the hosted engine broker and agent fail to start with
> error
> messages:
> ----------------------------------
> [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
> ovirt-ha-agent.service - oVirt Hosted Engine High Availability
> Monitoring Agent
> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
> Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> 20min ago
> Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
> (code=exited, status=1/FAILURE)
>
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr
> 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> File "/usr/lib64/python2.7/logging/__init__.py", line 902, in
> __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
> "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr
> 26
> 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> IOError: [Errno 6] No such device or address: '/dev/stdout'
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> [FAILED]
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> ovirt-ha-agent.service: control process exited, code=exited status=1
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to
> start oVirt Hosted Engine High Availability Monitoring Agent.
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> ovirt-ha-agent.service entered failed state.
> -------------------------------------
> And
> -------------------------------------
> [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
> ovirt-ha-broker.service - oVirt Hosted Engine High Availability
> Communications Broker
> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
> enabled)
> Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> 21min ago
> Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
> (code=exited, status=1/FAILURE)
>
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> hdlr = FileHandler(filename, mode)
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26
> 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26
> 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> IOError: [Errno 6] No such device or address: '/dev/stdout'
>
> Didi any clue?
> the log says it runs as root so I canrule that out
>
That's weird. Please check/post:
ls -l /dev/stdout
echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> [FAILED]
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> ovirt-ha-broker.service: control process exited, code=exited
> status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> Failed to start oVirt Hosted Engine High Availability Communications Broker.
> Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> ovirt-ha-broker.service entered failed state.
> ----------------------------------------
>
> The system is a CentOS 7 Setup with SeLinux switched off, no
> firewall or iptables. How can I find out which version of ovirt I'm
> running exactly ?
> I've had a lock at the logs and read through old bug reports.
>
>
> the rpm version of ovirt* will be enough I guess
>
>
> Thank you,
>
> Sven
>
>
>
>
> _______________________________________________
>
> Users mailing list
>
> Users@ovirt.org<mailto:Users@ovirt.org>
>
> http://lists.ovirt.org/mailman/listinfo/users
>
>
Thanks,
--
Didi
Best,
--
Didi
3:50 a.m.
Uh this really is weird.
The situation is clear though:
Broker dies when it tries to initialize logging (missing /dev/stdout ???)
Agent dies because it can't connect to the broker.
My /dev/stdout looks like this:
lrwxrwxrwx. 1 root root 15 Mar 30 17:29 /dev/stdout -> /proc/self/fd/1
And /proc/self/fd/1 is obviously related to the process. But I have an idea.
Can you check whether the /proc/self/fd/1 is there? It might be missing if
the broker closed its stdout during daemonizing.
--
Martin Sivák
msivak(a)redhat.com
Red Hat Czech
RHEV-M SLA / Brno, CZ
----- Original Message -----
Yes,
-----------------------
[root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl
start ovirt-ha-agent.service
Job for ovirt-ha-broker.service failed. See 'systemctl status
ovirt-ha-broker.service' and 'journalctl -xn' for details.
[root@ovirt-node2 ~]# journalctl -xn
-- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27 02:49:33
CDT. --
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
stream = open(self.baseFilename, self.mode)
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
IOError: [Errno 6] No such device or address: '/dev/stdout'
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
[FAILED]
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-broker.service: control process exited, code=exited status=1
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt
Hosted Engine High Availability Communications Broker.
-- Subject: Unit ovirt-ha-broker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit ovirt-ha-broker.service has failed.
--
-- The result is failed.
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-broker.service entered failed state.
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm
ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to
broker, the number of errors has exceeded the limit (
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR failed
to retrieve Hosted Engine HA info
Traceback (most recent
call last):
File
"/usr/share/vdsm/API.py",
line 1703, in
_getHaInfo
stats =
instance.get_all_stats()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
line 97, in
get_all_stats
with
broker.connection():
File
"/usr/lib64/python2.7/contextlib.py",
line 17, in
__enter__
return
self.gen.next()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 99, in
connection
self.connect()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 78, in connect
raise
BrokerConnectionError(error_msg)
BrokerConnectionError:
Failed to connect to
broker, the number of
errors has exceeded
the limit (5)
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not
found: Requested metadata element is not present
--------------------------------
-----Ursprüngliche Nachricht-----
Von: Yedidyah Bar David [mailto:didi@redhat.com]
Gesendet: Montag, 27. April 2015 09:46
An: Sven Achtelik; Martin Sivak
Cc: Roy Golan; users(a)ovirt.org
Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
> From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> To: "Yedidyah Bar David" <didi(a)redhat.com>
> Cc: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org
> Sent: Monday, April 27, 2015 10:34:13 AM
> Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional
> host
>
> Hi Did,
>
> results are
> ---------------------------
> [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr
> 26 09:14 /dev/stdout -> /proc/self/fd/1
> [root@ovirt-node2 ~]# echo test > /dev/stdout test
> ---------------------------
> Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
>
> Sven
>
>
> -----Ursprüngliche Nachricht-----
> Von: Yedidyah Bar David [mailto:didi@redhat.com]
> Gesendet: Montag, 27. April 2015 08:57
> An: Sven Achtelik
> Cc: Roy Golan; users(a)ovirt.org
> Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional
> host
>
>
>
> ----- Original Message -----
> > From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> > To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org,
"Yedidyah Bar
> > David" <didi(a)redhat.com>
> > Sent: Sunday, April 26, 2015 6:57:06 PM
> > Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
> >
> > On the node that fails to start the ha-broker and ha-agent I'm using:
> >
> > ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
> > @ovirt-3.5-pre
> > ovirt-host-deploy.noarch 1.3.1-1.el7
> > @ovirt-3.5
> > ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
> > @ovirt-3.5
> > ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
> > @ovirt-3.5-pre
> > ovirt-release35.noarch 003-1
> > @/ovirt-release35
> >
> >
> > Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im
> > Auftrag von Roy Golan
> > Gesendet: Sonntag, 26. April 2015 16:59
> > An: users(a)ovirt.org; Yedidyah Bar David
> > Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
> >
> > On 04/26/2015 05:38 PM, Sven Achtelik wrote:
> > Hi All,
> >
> > after a successful setup of hosted-engine on the first node I'm
> > having trouble completing it on an additional node. The Setup fails with:
> > ---------------------------------
> > [ INFO ] Waiting for the host to become operational in the engine.
> > This may take several minutes...
> > [ INFO ] Still waiting for VDSM host to become operational...
> > [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and
> > starting HA services [ ERROR ] Failed to execute stage 'Closing up':
> > Command '/bin/systemctl'
> > failed to execute
> > [ INFO ] Stage: Clean up
> > [ INFO ] Generating answer file
> >
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
> > [ INFO ] Stage: Pre-termination
> > [ INFO ] Stage: Termination
> > ---------------------------------
> > After that the node is added to the cluster and is operational from
> > the GUI, but the hosted engine broker and agent fail to start with
> > error
> > messages:
> > ----------------------------------
> > [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
> > ovirt-ha-agent.service - oVirt Hosted Engine High Availability
> > Monitoring Agent
> > Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service;
> > enabled)
> > Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> > 20min ago
> > Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
> > (code=exited, status=1/FAILURE)
> >
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr
> > 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > File "/usr/lib64/python2.7/logging/__init__.py", line 902, in
> > __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > systemd-ovirt-ha-agent[5373]:
> > StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
> > "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr
> > 26
> > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > IOError: [Errno 6] No such device or address: '/dev/stdout'
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > [FAILED]
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > ovirt-ha-agent.service: control process exited, code=exited status=1
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to
> > start oVirt Hosted Engine High Availability Monitoring Agent.
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> > ovirt-ha-agent.service entered failed state.
> > -------------------------------------
> > And
> > -------------------------------------
> > [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
> > ovirt-ha-broker.service - oVirt Hosted Engine High Availability
> > Communications Broker
> > Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
> > enabled)
> > Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> > 21min ago
> > Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
> > (code=exited, status=1/FAILURE)
> >
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > hdlr = FileHandler(filename, mode)
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26
> > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26
> > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > IOError: [Errno 6] No such device or address: '/dev/stdout'
> >
> > Didi any clue?
> > the log says it runs as root so I canrule that out
> >
>
> That's weird. Please check/post:
>
> ls -l /dev/stdout
> echo test > /dev/stdout
>
> It should be a symlink to /proc/self/fd/1 .
>
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > [FAILED]
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > ovirt-ha-broker.service: control process exited, code=exited
> > status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > Failed to start oVirt Hosted Engine High Availability Communications
> > Broker.
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> > ovirt-ha-broker.service entered failed state.
> > ----------------------------------------
> >
> > The system is a CentOS 7 Setup with SeLinux switched off, no
> > firewall or iptables. How can I find out which version of ovirt I'm
> > running exactly ?
> > I've had a lock at the logs and read through old bug reports.
> >
> >
> > the rpm version of ovirt* will be enough I guess
> >
> >
> > Thank you,
> >
> > Sven
> >
> >
> >
> >
> > _______________________________________________
> >
> > Users mailing list
> >
> > Users@ovirt.org<mailto:Users@ovirt.org>
> >
> > http://lists.ovirt.org/mailman/listinfo/users
> >
> >
>
> Thanks,
> --
> Didi
>
>
Best,
--
Didi
3:55 a.m.
----- Original Message -----
From: "Martin Sivak" <msivak(a)redhat.com>
To: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
Cc: "Yedidyah Bar David" <didi(a)redhat.com>, "Roy Golan"
<rgolan(a)redhat.com>, users(a)ovirt.org
Sent: Monday, April 27, 2015 11:50:30 AM
Subject: Re: AW: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Uh this really is weird.
The situation is clear though:
Broker dies when it tries to initialize logging (missing /dev/stdout ???)
Agent dies because it can't connect to the broker.
My /dev/stdout looks like this:
lrwxrwxrwx. 1 root root 15 Mar 30 17:29 /dev/stdout -> /proc/self/fd/1
And /proc/self/fd/1 is obviously related to the process. But I have an idea.
Can you check whether the /proc/self/fd/1 is there? It might be missing if
the broker closed its stdout during daemonizing.
If that's the problem, he can't see that - in his shell, /proc/self is its
shell's process.
Not sure how else to check without some small patch...
--
Martin Sivák
msivak(a)redhat.com
Red Hat Czech
RHEV-M SLA / Brno, CZ
----- Original Message -----
> Yes,
>
> -----------------------
> [root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl
> start ovirt-ha-agent.service
> Job for ovirt-ha-broker.service failed. See 'systemctl status
> ovirt-ha-broker.service' and 'journalctl -xn' for details.
> [root@ovirt-node2 ~]# journalctl -xn
> -- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27
> 02:49:33
> CDT. --
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
> File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
> stream = open(self.baseFilename, self.mode)
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
> IOError: [Errno 6] No such device or address: '/dev/stdout'
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
> [FAILED]
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]:
> ovirt-ha-broker.service: control process exited, code=exited status=1
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
> oVirt
> Hosted Engine High Availability Communications Broker.
> -- Subject: Unit ovirt-ha-broker.service has failed
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> --
> -- Unit ovirt-ha-broker.service has failed.
> --
> -- The result is failed.
> Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> ovirt-ha-broker.service entered failed state.
> Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm
> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to
> broker, the number of errors has exceeded the limit (
> Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR
> failed
> to retrieve Hosted Engine HA info
> Traceback (most
> recent
> call last):
> File
>
"/usr/share/vdsm/API.py",
> line 1703, in
> _getHaInfo
> stats =
> instance.get_all_stats()
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
> line 97, in
> get_all_stats
> with
> broker.connection():
> File
>
"/usr/lib64/python2.7/contextlib.py",
> line 17, in
> __enter__
> return
> self.gen.next()
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
> line 99, in
> connection
> self.connect()
> File
>
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
> line 78, in
> connect
> raise
>
BrokerConnectionError(error_msg)
> BrokerConnectionError:
> Failed to connect to
> broker, the number
> of
> errors has exceeded
> the limit (5)
> Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not
> found: Requested metadata element is not present
> --------------------------------
>
>
> -----Ursprüngliche Nachricht-----
> Von: Yedidyah Bar David [mailto:didi@redhat.com]
> Gesendet: Montag, 27. April 2015 09:46
> An: Sven Achtelik; Martin Sivak
> Cc: Roy Golan; users(a)ovirt.org
> Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional
> host
>
> ----- Original Message -----
> > From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> > To: "Yedidyah Bar David" <didi(a)redhat.com>
> > Cc: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org
> > Sent: Monday, April 27, 2015 10:34:13 AM
> > Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional
> > host
> >
> > Hi Did,
> >
> > results are
> > ---------------------------
> > [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr
> > 26 09:14 /dev/stdout -> /proc/self/fd/1
> > [root@ovirt-node2 ~]# echo test > /dev/stdout test
> > ---------------------------
> > Looks like everything is working fine.
>
> And it still fails with the same message when you restart ha daemons?
>
> Adding Martin.
>
> Weird.
>
> >
> > Sven
> >
> >
> > -----Ursprüngliche Nachricht-----
> > Von: Yedidyah Bar David [mailto:didi@redhat.com]
> > Gesendet: Montag, 27. April 2015 08:57
> > An: Sven Achtelik
> > Cc: Roy Golan; users(a)ovirt.org
> > Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional
> > host
> >
> >
> >
> > ----- Original Message -----
> > > From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> > > To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org,
"Yedidyah Bar
> > > David" <didi(a)redhat.com>
> > > Sent: Sunday, April 26, 2015 6:57:06 PM
> > > Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
> > >
> > > On the node that fails to start the ha-broker and ha-agent I'm using:
> > >
> > > ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
> > > @ovirt-3.5-pre
> > > ovirt-host-deploy.noarch 1.3.1-1.el7
> > > @ovirt-3.5
> > > ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
> > > @ovirt-3.5
> > > ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
> > > @ovirt-3.5-pre
> > > ovirt-release35.noarch 003-1
> > > @/ovirt-release35
> > >
> > >
> > > Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im
> > > Auftrag von Roy Golan
> > > Gesendet: Sonntag, 26. April 2015 16:59
> > > An: users(a)ovirt.org; Yedidyah Bar David
> > > Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
> > >
> > > On 04/26/2015 05:38 PM, Sven Achtelik wrote:
> > > Hi All,
> > >
> > > after a successful setup of hosted-engine on the first node I'm
> > > having trouble completing it on an additional node. The Setup fails
> > > with:
> > > ---------------------------------
> > > [ INFO ] Waiting for the host to become operational in the engine.
> > > This may take several minutes...
> > > [ INFO ] Still waiting for VDSM host to become operational...
> > > [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and
> > > starting HA services [ ERROR ] Failed to execute stage 'Closing
up':
> > > Command '/bin/systemctl'
> > > failed to execute
> > > [ INFO ] Stage: Clean up
> > > [ INFO ] Generating answer file
> > >
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
> > > [ INFO ] Stage: Pre-termination
> > > [ INFO ] Stage: Termination
> > > ---------------------------------
> > > After that the node is added to the cluster and is operational from
> > > the GUI, but the hosted engine broker and agent fail to start with
> > > error
> > > messages:
> > > ----------------------------------
> > > [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
> > > ovirt-ha-agent.service - oVirt Hosted Engine High Availability
> > > Monitoring Agent
> > > Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service;
> > > enabled)
> > > Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28
> > > CDT;
> > > 20min ago
> > > Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
> > > (code=exited, status=1/FAILURE)
> > >
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > > systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr
> > > 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > > File "/usr/lib64/python2.7/logging/__init__.py", line 902, in
> > > __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > > systemd-ovirt-ha-agent[5373]:
> > > StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> > > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
> > > "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr
> > > 26
> > > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > > stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> > > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > > IOError: [Errno 6] No such device or address: '/dev/stdout'
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > > systemd-ovirt-ha-agent[5373]:
> > > [FAILED]
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > > ovirt-ha-agent.service: control process exited, code=exited status=1
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to
> > > start oVirt Hosted Engine High Availability Monitoring Agent.
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> > > ovirt-ha-agent.service entered failed state.
> > > -------------------------------------
> > > And
> > > -------------------------------------
> > > [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
> > > ovirt-ha-broker.service - oVirt Hosted Engine High Availability
> > > Communications Broker
> > > Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
> > > enabled)
> > > Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28
> > > CDT;
> > > 21min ago
> > > Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker
> > > start
> > > (code=exited, status=1/FAILURE)
> > >
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > > systemd-ovirt-ha-broker[5359]:
> > > hdlr = FileHandler(filename, mode)
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > > systemd-ovirt-ha-broker[5359]:
> > > File "/usr/lib64/python2.7/logging/__init__.py", line ...it__
Apr 26
> > > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > > StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> > > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > > File "/usr/lib64/python2.7/logging/__init__.py", line ...open
Apr 26
> > > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > > stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> > > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > > IOError: [Errno 6] No such device or address: '/dev/stdout'
> > >
> > > Didi any clue?
> > > the log says it runs as root so I canrule that out
> > >
> >
> > That's weird. Please check/post:
> >
> > ls -l /dev/stdout
> > echo test > /dev/stdout
> >
> > It should be a symlink to /proc/self/fd/1 .
> >
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > > systemd-ovirt-ha-broker[5359]:
> > > [FAILED]
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > > ovirt-ha-broker.service: control process exited, code=exited
> > > status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > > Failed to start oVirt Hosted Engine High Availability Communications
> > > Broker.
> > > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> > > ovirt-ha-broker.service entered failed state.
> > > ----------------------------------------
> > >
> > > The system is a CentOS 7 Setup with SeLinux switched off, no
> > > firewall or iptables. How can I find out which version of ovirt I'm
> > > running exactly ?
> > > I've had a lock at the logs and read through old bug reports.
> > >
> > >
> > > the rpm version of ovirt* will be enough I guess
> > >
> > >
> > > Thank you,
> > >
> > > Sven
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > >
> > > Users mailing list
> > >
> > > Users@ovirt.org<mailto:Users@ovirt.org>
> > >
> > > http://lists.ovirt.org/mailman/listinfo/users
> > >
> > >
> >
> > Thanks,
> > --
> > Didi
> >
> >
>
> Best,
> --
> Didi
>
>
--
Didi
4:19 a.m.
Hi Martin,
on my root login things look like this:
[root@ovirt-node2 ~]# ls -lah /proc/self/fd/1
lrwx------ 1 root root 64 Apr 27 04:02 /proc/self/fd/1 -> /dev/pts/1
Sven
-----Ursprüngliche Nachricht-----
Von: Martin Sivak [mailto:msivak@redhat.com]
Gesendet: Montag, 27. April 2015 10:51
An: Sven Achtelik
Cc: Yedidyah Bar David; Roy Golan; users(a)ovirt.org
Betreff: Re: AW: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Uh this really is weird.
The situation is clear though:
Broker dies when it tries to initialize logging (missing /dev/stdout ???) Agent dies
because it can't connect to the broker.
My /dev/stdout looks like this:
lrwxrwxrwx. 1 root root 15 Mar 30 17:29 /dev/stdout -> /proc/self/fd/1
And /proc/self/fd/1 is obviously related to the process. But I have an idea.
Can you check whether the /proc/self/fd/1 is there? It might be missing if the broker
closed its stdout during daemonizing.
--
Martin Sivák
msivak(a)redhat.com
Red Hat Czech
RHEV-M SLA / Brno, CZ
----- Original Message -----
Yes,
-----------------------
[root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service &&
systemctl start ovirt-ha-agent.service Job for ovirt-ha-broker.service
failed. See 'systemctl status ovirt-ha-broker.service' and 'journalctl
-xn' for details.
[root@ovirt-node2 ~]# journalctl -xn
-- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27
02:49:33 CDT. -- Apr 27 02:49:27 ovirt-node2.mgmt.asl.local
systemd-ovirt-ha-broker[29068]:
File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
stream = open(self.baseFilename, self.mode) Apr 27 02:49:27
ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
IOError: [Errno 6] No such device or address: '/dev/stdout'
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]:
[FAILED]
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]:
ovirt-ha-broker.service: control process exited, code=exited status=1
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start
oVirt Hosted Engine High Availability Communications Broker.
-- Subject: Unit ovirt-ha-broker.service has failed
-- Defined-By: systemd
-- Support:
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit ovirt-ha-broker.service has failed.
--
-- The result is failed.
Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit
ovirt-ha-broker.service entered failed state.
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm
ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to
connect to broker, the number of errors has exceeded the limit ( Apr
27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR
failed to retrieve Hosted Engine HA info
Traceback (most recent
call last):
File
"/usr/share/vdsm/API.py",
line 1703, in
_getHaInfo
stats =
instance.get_all_stats()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
line 97, in
get_all_stats
with
broker.connection():
File
"/usr/lib64/python2.7/contextlib.py",
line 17, in
__enter__
return
self.gen.next()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 99, in
connection
self.connect()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 78, in connect
raise
BrokerConnectionError(error_msg)
BrokerConnectionError:
Failed to connect to
broker, the number of
errors has exceeded
the limit (5)
Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata
not
found: Requested metadata element is not present
--------------------------------
-----Ursprüngliche Nachricht-----
Von: Yedidyah Bar David [mailto:didi@redhat.com]
Gesendet: Montag, 27. April 2015 09:46
An: Sven Achtelik; Martin Sivak
Cc: Roy Golan; users(a)ovirt.org
Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue
additional host
----- Original Message -----
> From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> To: "Yedidyah Bar David" <didi(a)redhat.com>
> Cc: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org
> Sent: Monday, April 27, 2015 10:34:13 AM
> Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional
> host
>
> Hi Did,
>
> results are
> ---------------------------
> [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15
> Apr
> 26 09:14 /dev/stdout -> /proc/self/fd/1
> [root@ovirt-node2 ~]# echo test > /dev/stdout test
> ---------------------------
> Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
>
> Sven
>
>
> -----Ursprüngliche Nachricht-----
> Von: Yedidyah Bar David [mailto:didi@redhat.com]
> Gesendet: Montag, 27. April 2015 08:57
> An: Sven Achtelik
> Cc: Roy Golan; users(a)ovirt.org
> Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional
> host
>
>
>
> ----- Original Message -----
> > From: "Sven Achtelik" <Sven.Achtelik(a)mailpool.us>
> > To: "Roy Golan" <rgolan(a)redhat.com>, users(a)ovirt.org,
"Yedidyah
> > Bar David" <didi(a)redhat.com>
> > Sent: Sunday, April 26, 2015 6:57:06 PM
> > Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional
> > host
> >
> > On the node that fails to start the ha-broker and ha-agent I'm using:
> >
> > ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos
> > @ovirt-3.5-pre
> > ovirt-host-deploy.noarch 1.3.1-1.el7
> > @ovirt-3.5
> > ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos
> > @ovirt-3.5
> > ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos
> > @ovirt-3.5-pre
> > ovirt-release35.noarch 003-1
> > @/ovirt-release35
> >
> >
> > Von: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] Im
> > Auftrag von Roy Golan
> > Gesendet: Sonntag, 26. April 2015 16:59
> > An: users(a)ovirt.org; Yedidyah Bar David
> > Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional
> > host
> >
> > On 04/26/2015 05:38 PM, Sven Achtelik wrote:
> > Hi All,
> >
> > after a successful setup of hosted-engine on the first node I'm
> > having trouble completing it on an additional node. The Setup fails with:
> > ---------------------------------
> > [ INFO ] Waiting for the host to become operational in the engine.
> > This may take several minutes...
> > [ INFO ] Still waiting for VDSM host to become operational...
> > [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and
> > starting HA services [ ERROR ] Failed to execute stage 'Closing up':
> > Command '/bin/systemctl'
> > failed to execute
> > [ INFO ] Stage: Clean up
> > [ INFO ] Generating answer file
> >
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
> > [ INFO ] Stage: Pre-termination
> > [ INFO ] Stage: Termination
> > ---------------------------------
> > After that the node is added to the cluster and is operational
> > from the GUI, but the hosted engine broker and agent fail to
> > start with error
> > messages:
> > ----------------------------------
> > [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
> > ovirt-ha-agent.service - oVirt Hosted Engine High Availability
> > Monitoring Agent
> > Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service;
> > enabled)
> > Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> > 20min ago
> > Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start
> > (code=exited, status=1/FAILURE)
> >
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode)
> > Apr
> > 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > File "/usr/lib64/python2.7/logging/__init__.py", line 902, in
> > __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local
> > systemd-ovirt-ha-agent[5373]:
> > StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File
> > "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr
> > 26
> > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > IOError: [Errno 6] No such device or address: '/dev/stdout'
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:
> > [FAILED]
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > ovirt-ha-agent.service: control process exited, code=exited
> > status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > Failed to start oVirt Hosted Engine High Availability Monitoring Agent.
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> > ovirt-ha-agent.service entered failed state.
> > -------------------------------------
> > And
> > -------------------------------------
> > [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
> > ovirt-ha-broker.service - oVirt Hosted Engine High Availability
> > Communications Broker
> > Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;
> > enabled)
> > Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT;
> > 21min ago
> > Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start
> > (code=exited, status=1/FAILURE)
> >
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > hdlr = FileHandler(filename, mode) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr
> > 26
> > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > StreamHandler.__init__(self, self._open()) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr
> > 26
> > 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > stream = open(self.baseFilename, self.mode) Apr 26 08:00:28
> > ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > IOError: [Errno 6] No such device or address: '/dev/stdout'
> >
> > Didi any clue?
> > the log says it runs as root so I canrule that out
> >
>
> That's weird. Please check/post:
>
> ls -l /dev/stdout
> echo test > /dev/stdout
>
> It should be a symlink to /proc/self/fd/1 .
>
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]:
> > [FAILED]
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > ovirt-ha-broker.service: control process exited, code=exited
> > status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:
> > Failed to start oVirt Hosted Engine High Availability
> > Communications Broker.
> > Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit
> > ovirt-ha-broker.service entered failed state.
> > ----------------------------------------
> >
> > The system is a CentOS 7 Setup with SeLinux switched off, no
> > firewall or iptables. How can I find out which version of ovirt
> > I'm running exactly ?
> > I've had a lock at the logs and read through old bug reports.
> >
> >
> > the rpm version of ovirt* will be enough I guess
> >
> >
> > Thank you,
> >
> > Sven
> >
> >
> >
> >
> > _______________________________________________
> >
> > Users mailing list
> >
> > Users@ovirt.org<mailto:Users@ovirt.org>
> >
> > http://lists.ovirt.org/mailman/listinfo/users
> >
> >
>
> Thanks,
> --
> Didi
>
>
Best,
--
Didi
3280
days inactive
3281
days old
9 comments
4 participants
participants (4)
-
Martin Sivak -
Roy Golan -
Sven Achtelik -
Yedidyah Bar David