Hosted Engine-Setup issue additional host
Process: 5373 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-agent s= tart (code=3Dexited, status=3D1/FAILURE)<o:p></o:p></span></p><p class=3DMs= oNormal><span lang=3DEN-US><o:p> </o:p></span></p><p class=3DMsoNormal= <span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovir= t-ha-agent[5373]: hdlr =3D FileHandler(filename, mode)<o:p></o:p></span></p= <p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.a= sl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logg= ing/__init__.py", line 902, in __init__<o:p></o:p></span></p><p class= =3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local = systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open())<o:=
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: I= OError: [Errno 6] No such device or address: '/dev/stdout'<o:p></o:p></span= </p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mg= mt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED]<o:p></o:p></span></p><p= class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.= local systemd[1]: ovirt-ha-agent.service: control process exited, code=3Dex= ited status=3D1<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-U= S>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oV= irt Hosted Engine High Availability Monitoring Agent.<o:p></o:p></span></p>= <p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.as= l.local systemd[1]: Unit ovirt-ha-agent.service entered failed state.<o:p><= /o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>-------------------= ------------------<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DE= N-US>And <o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>----= ---------------------------------<o:p></o:p></span></p><p class=3DMsoNormal= <span lang=3DEN-US>[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker<= o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>ovirt-ha-broke= r.service - oVirt Hosted Engine High Availability Communications Broker<o:p= </o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US> Load= ed: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled)<o:p><= /o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US> Active= : failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago<o= :p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US> Process:= 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-broker start (code=3Dex= ited, status=3D1/FAILURE)<o:p></o:p></span></p><p class=3DMsoNormal><span l= ang=3DEN-US><o:p> </o:p></span></p><p class=3DMsoNormal><span lang=3DE= N-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[535= 9]: hdlr =3D FileHandler(filename, mode)<o:p></o:p></span></p><p class=3DMs= oNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local syste= md-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.=
<span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovir= t-ha-broker[5359]: [FAILED]<o:p></o:p></span></p><p class=3DMsoNormal><span= lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-= ha-broker.service: control process exited, code=3Dexited status=3D1<o:p></o= :p></span></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt= -node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High = Availability Communications Broker.<o:p></o:p></span></p><p class=3DMsoNorm= al><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]= : Unit ovirt-ha-broker.service entered failed state.<o:p></o:p></span></p><=
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2SOCRATESasl_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi All, after a successful setup of hosted-engine on the first node I'm having trou= ble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This ma= y take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' fa= iled to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answer= s/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI= , but the hosted engine broker and agent fail to start with error messages= : ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring A= gent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20= min ago Process: 5373 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-agent start (= code=3Dexited, status=3D1/FAILURE) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hd= lr =3D FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi= le "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: St= reamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi= le "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: st= ream =3D open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IO= Error: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [F= AILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.servi= ce: control process exited, code=3Dexited status=3D1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir= t Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.= service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communicati= ons Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled= ) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21= min ago Process: 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-broker start = (code=3Dexited, status=3D1/FAILURE) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: h= dlr =3D FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F= ile "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: S= treamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F= ile "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: s= tream =3D open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: I= OError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [= FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.serv= ice: control process exited, code=3Dexited status=3D1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir= t Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker= .service entered failed state. ---------------------------------------- The system is a CentOS 7 Setup with SeLinux switched off, no firewall or ip= tables. How can I find out which version of ovirt I'm running exactly ? I'= ve had a lock at the logs and read through old bug reports. Thank you, Sven --_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2SOCRATESasl_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content= =3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator content=3D"Micros= oft Word 15 (filtered medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} span.E-MailFormatvorlage17 {mso-style-type:personal-compose; font-family:"Calibri",sans-serif; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 2.0cm 70.85pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body lang=3DDE link=3D"#0563C1" v= link=3D"#954F72"><div class=3DWordSection1><p class=3DMsoNormal><span lang= =3DEN-US>Hi All,<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-= US><o:p> </o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>afte= r a successful setup of hosted-engine on the first node I’m having tr= ouble completing it on an additional node. The Setup fails with:<o:p></o:p>= </span></p><p class=3DMsoNormal><span lang=3DEN-US>------------------------= ---------<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>[ IN= FO ] Waiting for the host to become operational in the engine. This m= ay take several minutes...<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US>[ INFO ] Still waiting for VDSM host to become operation= al...<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO&n= bsp; ] The VDSM Host is now operational<o:p></o:p></span></p><p class=3DMso= Normal><span lang=3DEN-US>[ INFO ] Enabling and starting HA services<= o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>[ ERROR ] Fail= ed to execute stage 'Closing up': Command '/bin/systemctl' failed to execut= e<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO = ] Stage: Clean up<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DE= N-US>[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-se= tup/answers/answers-20150426080028.conf'<o:p></o:p></span></p><p class=3DMs= oNormal><span lang=3DEN-US>[ INFO ] Stage: Pre-termination<o:p></o:p>= </span></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO ] Stage: Te= rmination<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>----= -----------------------------<o:p></o:p></span></p><p class=3DMsoNormal><sp= an lang=3DEN-US>After that the node is added to the cluster and is operatio= nal from the GUI, but the hosted engine broker and agent fail to star= t with error messages: <o:p></o:p></span></p><p class=3DMsoNormal><span lan= g=3DEN-US>----------------------------------<o:p></o:p></span></p><p class= =3DMsoNormal><span lang=3DEN-US>[root@ovirt-node2 ~]# systemctl status ovir= t-ha-agent.service -l<o:p></o:p></span></p><p class=3DMsoNormal><span lang= =3DEN-US>ovirt-ha-agent.service - oVirt Hosted Engine High Availability Mon= itoring Agent<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>= Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service= ; enabled)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>&nb= sp; Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 = CDT; 20min ago<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US= p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 = ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/li= b64/python2.7/logging/__init__.py", line 925, in _open<o:p></o:p></spa= n></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.m= gmt.asl.local systemd-ovirt-ha-agent[5373]: stream =3D open(self.baseFilena= me, self.mode)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US= py", line ...it__<o:p></o:p></span></p><p class=3DMsoNormal><span lang= =3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker= [5359]: StreamHandler.__init__(self, self._open())<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.l= ocal systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging= /__init__.py", line ...open<o:p></o:p></span></p><p class=3DMsoNormal>= <span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt= -ha-broker[5359]: stream =3D open(self.baseFilename, self.mode)<o:p></o:p><= /span></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-nod= e2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such= device or address: '/dev/stdout'<o:p></o:p></span></p><p class=3DMsoNormal= p class=3DMsoNormal><span lang=3DEN-US>------------------------------------= ----<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US><o:p>&nbs= p;</o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US>The system is a = CentOS 7 Setup with SeLinux switched off, no firewall or iptables. Ho= w can I find out which version of ovirt I’m running exactly ? I’= ;ve had a lock at the logs and read through old bug reports.<o:p></o:p></sp= an></p><p class=3DMsoNormal><span lang=3DEN-US><o:p> </o:p></span></p>= <p class=3DMsoNormal><span lang=3DEN-US>Thank you, <o:p></o:p></span></p><p= class=3DMsoNormal><span lang=3DEN-US><o:p> </o:p></span></p><p class= =3DMsoNormal><span lang=3DEN-US>Sven <o:p></o:p></span></p></div></body></h= tml>= --_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2SOCRATESasl_--
This is a multi-part message in MIME format. --------------070001000400050301000408 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit On 04/26/2015 05:38 PM, Sven Achtelik wrote:
Hi All,
after a successful setup of hosted-engine on the first node Im having trouble completing it on an additional node. The Setup fails with:
---------------------------------
[ INFO ] Waiting for the host to become operational in the engine. This may take several minutes...
[ INFO ] Still waiting for VDSM host to become operational...
[ INFO ] The VDSM Host is now operational
[ INFO ] Enabling and starting HA services
[ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
---------------------------------
After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages:
----------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l
ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago
Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state.
-------------------------------------
And
-------------------------------------
[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker
ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker
Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled)
Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago
Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open())
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED]
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker.
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state.
----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt Im running exactly ? Ive had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--------------070001000400050301000408 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#FFFFFF"> <div class="moz-cite-prefix">On 04/26/2015 05:38 PM, Sven Achtelik wrote:<br> </div> <blockquote cite="mid:BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2@SOCRATES.asl.local" type="cite"> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} span.E-MailFormatvorlage17 {mso-style-type:personal-compose; font-family:"Calibri",sans-serif; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 2.0cm 70.85pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext="edit" spidmax="1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext="edit"> <o:idmap v:ext="edit" data="1" /> </o:shapelayout></xml><![endif]--> <div class="WordSection1"> <p class="MsoNormal"><span lang="EN-US">Hi All,<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US">after a successful setup of hosted-engine on the first node Im having trouble completing it on an additional node. The Setup fails with:<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">---------------------------------<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Waiting for the host to become operational in the engine. This may take several minutes...<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Still waiting for VDSM host to become operational...<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] The VDSM Host is now operational<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Enabling and starting HA services<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Stage: Clean up<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf'<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Stage: Pre-termination<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[ INFO ] Stage: Termination<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">---------------------------------<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: <o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">----------------------------------<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"> Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"> Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open())<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout'<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED]<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">-------------------------------------<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">And <o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">-------------------------------------<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">[root@ovirt-node2 ~]# systemctl status ovirt-ha-broker<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"> Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"> Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open())<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode)<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'</span></p> </div> </blockquote> <br> Didi any clue? <br> the log says it runs as root so I canrule that out<br> <blockquote cite="mid:BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2@SOCRATES.asl.local" type="cite"> <div class="WordSection1"> <p class="MsoNormal"><span lang="EN-US"><o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED]<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US">----------------------------------------<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US">The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt Im running exactly ? Ive had a lock at the logs and read through old bug reports.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> </div> </blockquote> <br> the rpm version of ovirt* will be enough I guess<br> <br> <blockquote cite="mid:BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB2@SOCRATES.asl.local" type="cite"> <div class="WordSection1"> <p class="MsoNormal"><span lang="EN-US">Thank you, <o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US">Sven <o:p></o:p></span></p> </div> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> </body> </html> --------------070001000400050301000408--
<span lang=3DEN-US> Loaded: loaded (/usr/lib/systemd/system/ov= irt-ha-agent.service; enabled)</span><o:p></o:p></p><p class=3DMsoNormal><s=
</o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-no= de2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(sel= f, self._open())</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-= US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]:= File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _o=
<span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovir= t-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdou= t'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:0= 0:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED]</spa= n><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ov= irt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control proces= s exited, code=3Dexited status=3D1</span><o:p></o:p></p><p class=3DMsoNorma= l><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]:= Failed to start oVirt Hosted Engine High Availability Monitoring Agent.</s=
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB7SOCRATESasl_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On the node that fails to start the ha-broker and ha-agent I'm using: ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-= 3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 = @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovir= t-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-= pre ovirt-release35.noarch 003-1 = @/ovirt-release35 Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag vo= n Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All, after a successful setup of hosted-engine on the first node I'm having trou= ble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This ma= y take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' fa= iled to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answer= s/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI= , but the hosted engine broker and agent fail to start with error messages= : ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring A= gent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20= min ago Process: 5373 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-agent start (= code=3Dexited, status=3D1/FAILURE) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hd= lr =3D FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi= le "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: St= reamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: Fi= le "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: st= ream =3D open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IO= Error: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [F= AILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.servi= ce: control process exited, code=3Dexited status=3D1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir= t Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.= service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communicati= ons Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled= ) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21= min ago Process: 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-broker start = (code=3Dexited, status=3D1/FAILURE) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: h= dlr =3D FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F= ile "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: S= treamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: F= ile "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: s= tream =3D open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: I= OError: [Errno 6] No such device or address: '/dev/stdout' Didi any clue? the log says it runs as root so I canrule that out Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [= FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.serv= ice: control process exited, code=3Dexited status=3D1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVir= t Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker= .service entered failed state. ---------------------------------------- The system is a CentOS 7 Setup with SeLinux switched off, no firewall or ip= tables. How can I find out which version of ovirt I'm running exactly ? I'= ve had a lock at the logs and read through old bug reports. the rpm version of ovirt* will be enough I guess Thank you, Sven _______________________________________________ Users mailing list Users@ovirt.org<mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users --_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB7SOCRATESasl_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content= =3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator content=3D"Micros= oft Word 15 (filtered medium)"><style><!-- /* Font Definitions */ @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri",sans-serif; color:black; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} pre {mso-style-priority:99; mso-style-link:"HTML Vorformatiert Zchn"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New"; color:black;} span.E-MailFormatvorlage17 {mso-style-type:personal; font-family:"Calibri",sans-serif; color:windowtext;} span.HTMLVorformatiertZchn {mso-style-name:"HTML Vorformatiert Zchn"; mso-style-priority:99; mso-style-link:"HTML Vorformatiert"; font-family:"Consolas",serif; color:black; mso-fareast-language:EN-US;} span.E-MailFormatvorlage20 {mso-style-type:personal-reply; font-family:"Calibri",sans-serif; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 2.0cm 70.85pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body bgcolor=3Dwhite lang=3DDE li= nk=3D"#0563C1" vlink=3D"#954F72"><div class=3DWordSection1><p class=3DMsoNo= rmal><span lang=3DEN-US style=3D'color:#1F497D'>On the node that fails to s= tart the ha-broker and ha-agent I’m using: <o:p></o:p></span></p><p c= lass=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'><o:p> </o:= p></span></p><p class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D= '>ovirt-engine-sdk-python.noarch = 3.5.2.1-1.el7.centos @ovir= t-3.5-pre<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US styl= e=3D'color:#1F497D'>ovirt-host-deploy.noarch = &n= bsp; 1.3.1-1.el7 = ; &= nbsp;@ovirt-3.5<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-U= S style=3D'color:#1F497D'>ovirt-hosted-engine-ha.noarch &= nbsp; &nbs= p; 1.2.5-1.el7.centos @ovirt-3.5<o:p></o:p></span><= /p><p class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>ovirt-ho= sted-engine-setup.noarch 1.2.3-1.= el7.centos @ovirt-3.5-pre<o:p></o:p></span></= p><p class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>ovirt-rel= ease35.noarch &= nbsp; &nb= sp; 003-1 = &n= bsp; @/ovirt-release35<o:p></o:p></span>= </p><p class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'><o:p>&n= bsp;</o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US><o:p> </o= :p></span></p><div><div style=3D'border:none;border-top:solid #E1E1E1 1.0pt= ;padding:3.0pt 0cm 0cm 0cm'><p class=3DMsoNormal><b><span style=3D'color:wi= ndowtext;mso-fareast-language:DE'>Von:</span></b><span style=3D'color:windo= wtext;mso-fareast-language:DE'> users-bounces@ovirt.org [mailto:users-bounc= es@ovirt.org] <b>Im Auftrag von </b>Roy Golan<br><b>Gesendet:</b> Sonntag, = 26. April 2015 16:59<br><b>An:</b> users@ovirt.org; Yedidyah Bar David<br><= b>Betreff:</b> Re: [ovirt-users] Hosted Engine-Setup issue additional host<= o:p></o:p></span></p></div></div><p class=3DMsoNormal><o:p> </o:p></p>= <div><p class=3DMsoNormal>On 04/26/2015 05:38 PM, Sven Achtelik wrote:<span= style=3D'font-size:12.0pt;mso-fareast-language:DE'><o:p></o:p></span></p><= /div><blockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'><p class=3D= MsoNormal><span lang=3DEN-US>Hi All,</span><o:p></o:p></p><p class=3DMsoNor= mal><span lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal><sp= an lang=3DEN-US>after a successful setup of hosted-engine on the first node= I’m having trouble completing it on an additional node. The Setup fa= ils with:</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>----= -----------------------------</span><o:p></o:p></p><p class=3DMsoNormal><sp= an lang=3DEN-US>[ INFO ] Waiting for the host to become operational i= n the engine. This may take several minutes...</span><o:p></o:p></p><p clas= s=3DMsoNormal><span lang=3DEN-US>[ INFO ] Still waiting for VDSM host= to become operational...</span><o:p></o:p></p><p class=3DMsoNormal><span l= ang=3DEN-US>[ INFO ] The VDSM Host is now operational</span><o:p></o:= p></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO ] Enabling and s= tarting HA services</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3D= EN-US>[ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemc= tl' failed to execute</span><o:p></o:p></p><p class=3DMsoNormal><span lang= =3DEN-US>[ INFO ] Stage: Clean up</span><o:p></o:p></p><p class=3DMso= Normal><span lang=3DEN-US>[ INFO ] Generating answer file '/var/lib/o= virt-hosted-engine-setup/answers/answers-20150426080028.conf'</span><o:p></= o:p></p><p class=3DMsoNormal><span lang=3DEN-US>[ INFO ] Stage: Pre-t= ermination</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>[ I= NFO ] Stage: Termination</span><o:p></o:p></p><p class=3DMsoNormal><s= pan lang=3DEN-US>---------------------------------</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>After that the node is added to the cl= uster and is operational from the GUI, but the hosted engine broker a= nd agent fail to start with error messages: </span><o:p></o:p></p><p class= =3DMsoNormal><span lang=3DEN-US>----------------------------------</span><o= :p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>[root@ovirt-node2 ~]# = systemctl status ovirt-ha-agent.service -l</span><o:p></o:p></p><p class=3D= MsoNormal><span lang=3DEN-US>ovirt-ha-agent.service - oVirt Hosted Engine H= igh Availability Monitoring Agent</span><o:p></o:p></p><p class=3DMsoNormal= pan lang=3DEN-US> Active: failed (Result: exit-code) since Sun = 2015-04-26 08:00:28 CDT; 20min ago</span><o:p></o:p></p><p class=3DMsoNorma= l><span lang=3DEN-US> Process: 5373 ExecStart=3D/usr/lib/systemd/syst= emd-ovirt-ha-agent start (code=3Dexited, status=3D1/FAILURE)</span><o:p></o= :p></p><p class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p>= <p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.as= l.local systemd-ovirt-ha-agent[5373]: hdlr =3D FileHandler(filename, mode)<= /span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:2= 8 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/= lib64/python2.7/logging/__init__.py", line 902, in __init__</span><o:p= pen</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:= 00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream =3D o= pen(self.baseFilename, self.mode)</span><o:p></o:p></p><p class=3DMsoNormal= pan><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 = ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered = failed state.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>= -------------------------------------</span><o:p></o:p></p><p class=3DMsoNo= rmal><span lang=3DEN-US>And </span><o:p></o:p></p><p class=3DMsoNormal><spa= n lang=3DEN-US>-------------------------------------</span><o:p></o:p></p><= p class=3DMsoNormal><span lang=3DEN-US>[root@ovirt-node2 ~]# systemctl stat= us ovirt-ha-broker</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DE= N-US>ovirt-ha-broker.service - oVirt Hosted Engine High Availability Commun= ications Broker</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-U= S> Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.serv= ice; enabled)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>= Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:= 28 CDT; 21min ago</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN= -US> Process: 5359 ExecStart=3D/usr/lib/systemd/systemd-ovirt-ha-brok= er start (code=3Dexited, status=3D1/FAILURE)</span><o:p></o:p></p><p class= =3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoN= ormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd= -ovirt-ha-broker[5359]: hdlr =3D FileHandler(filename, mode)</span><o:p></o= :p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.= mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2= .7/logging/__init__.py", line ...it__</span><o:p></o:p></p><p class=3D= MsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local sys= temd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open())</spa= n><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ov= irt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib= 64/python2.7/logging/__init__.py", line ...open</span><o:p></o:p></p><= p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl= .local systemd-ovirt-ha-broker[5359]: stream =3D open(self.baseFilename, se= lf.mode)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 2= 6 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOErro= r: [Errno 6] No such device or address: '/dev/stdout'</span><o:p></o:p></p>= </blockquote><p class=3DMsoNormal><span style=3D'font-size:12.0pt;font-fami= ly:"Times New Roman",serif;mso-fareast-language:DE'><br>Didi any clue? <br>= the log says it runs as root so I canrule that out<br><br><o:p></o:p></span=
</p><blockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'><p class=3D= MsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local sys= temd-ovirt-ha-broker[5359]: [FAILED]</span><o:p></o:p></p><p class=3DMsoNor= mal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1= ]: ovirt-ha-broker.service: control process exited, code=3Dexited status=3D= 1</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Apr 26 08:00= :28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Eng= ine High Availability Communications Broker.</span><o:p></o:p></p><p class= =3DMsoNormal><span lang=3DEN-US>Apr 26 08:00:28 ovirt-node2.mgmt.asl.local = systemd[1]: Unit ovirt-ha-broker.service entered failed state.</span><o:p><= /o:p></p><p class=3DMsoNormal><span lang=3DEN-US>--------------------------= --------------</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US= </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>The sy= stem is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables= . How can I find out which version of ovirt I’m running exactly= ? I’ve had a lock at the logs and read through old bug reports.</spa= n><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US> </span><o:p>= </o:p></p></blockquote><p class=3DMsoNormal><span style=3D'font-size:12.0pt= ;font-family:"Times New Roman",serif;mso-fareast-language:DE'><br>the rpm v= ersion of ovirt* will be enough I guess<br><br><br><o:p></o:p></span></p><b= lockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'><p class=3DMsoNorm= al><span lang=3DEN-US>Thank you, </span><o:p></o:p></p><p class=3DMsoNormal= <span lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>Sven </span><o:p></o:p></p><p class=3DMsoNormal><span style=3D= 'font-size:12.0pt;font-family:"Times New Roman",serif;mso-fareast-language:= DE'><br><br><br><o:p></o:p></span></p><pre>________________________________= _______________<o:p></o:p></pre><pre>Users mailing list<o:p></o:p></pre><pr= e><a href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre><p= re><a href=3D"http://lists.ovirt.org/mailman/listinfo/users">http://lists.o= virt.org/mailman/listinfo/users</a><o:p></o:p></pre></blockquote><p class= =3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times New Roman",= serif;mso-fareast-language:DE'><o:p> </o:p></span></p></div></body></h= tml>=
--_000_BFAB40933B3367488CE6299BAF8592D1010C9C2A4AB7SOCRATESasl_--
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post: ls -l /dev/stdout echo test > /dev/stdout It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
Hi Did, results are --------------------------- [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1 [root@ovirt-node2 ~]# echo test > /dev/stdout test --------------------------- Looks like everything is working fine. Sven -----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 08:57 An: Sven Achtelik Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host ----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post: ls -l /dev/stdout echo test > /dev/stdout It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Yedidyah Bar David" <didi@redhat.com> Cc: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org Sent: Monday, April 27, 2015 10:34:13 AM Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Hi Did,
results are --------------------------- [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1 [root@ovirt-node2 ~]# echo test > /dev/stdout test --------------------------- Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons? Adding Martin. Weird.
Sven
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 08:57 An: Sven Achtelik Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
Best, -- Didi
Yes, ----------------------- [root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl start ovirt-ha-agent.service Job for ovirt-ha-broker.service failed. See 'systemctl status ovirt-ha-broker.service' and 'journalctl -xn' for details. [root@ovirt-node2 ~]# journalctl -xn -- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27 02:49:33 CDT. -- Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: stream = open(self.baseFilename, self.mode) Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: [FAILED] Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. -- Subject: Unit ovirt-ha-broker.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ovirt-ha-broker.service has failed. -- -- The result is failed. Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to broker, the number of errors has exceeded the limit ( Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR failed to retrieve Hosted Engine HA info Traceback (most recent call last): File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo stats = instance.get_all_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 97, in get_all_stats with broker.connection(): File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 99, in connection self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 78, in connect raise BrokerConnectionError(error_msg) BrokerConnectionError: Failed to connect to broker, the number of errors has exceeded the limit (5) Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not found: Requested metadata element is not present -------------------------------- -----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 09:46 An: Sven Achtelik; Martin Sivak Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host ----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Yedidyah Bar David" <didi@redhat.com> Cc: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org Sent: Monday, April 27, 2015 10:34:13 AM Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Hi Did,
results are --------------------------- [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1 [root@ovirt-node2 ~]# echo test > /dev/stdout test --------------------------- Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons? Adding Martin. Weird.
Sven
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 08:57 An: Sven Achtelik Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
Best, -- Didi
Uh this really is weird. The situation is clear though: Broker dies when it tries to initialize logging (missing /dev/stdout ???) Agent dies because it can't connect to the broker. My /dev/stdout looks like this: lrwxrwxrwx. 1 root root 15 Mar 30 17:29 /dev/stdout -> /proc/self/fd/1 And /proc/self/fd/1 is obviously related to the process. But I have an idea. Can you check whether the /proc/self/fd/1 is there? It might be missing if the broker closed its stdout during daemonizing. -- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message -----
Yes,
----------------------- [root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl start ovirt-ha-agent.service Job for ovirt-ha-broker.service failed. See 'systemctl status ovirt-ha-broker.service' and 'journalctl -xn' for details. [root@ovirt-node2 ~]# journalctl -xn -- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27 02:49:33 CDT. -- Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: stream = open(self.baseFilename, self.mode) Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: [FAILED] Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. -- Subject: Unit ovirt-ha-broker.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ovirt-ha-broker.service has failed. -- -- The result is failed. Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to broker, the number of errors has exceeded the limit ( Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR failed to retrieve Hosted Engine HA info Traceback (most recent call last): File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo stats = instance.get_all_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 97, in get_all_stats with broker.connection(): File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 99, in connection self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 78, in connect raise BrokerConnectionError(error_msg) BrokerConnectionError: Failed to connect to broker, the number of errors has exceeded the limit (5) Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not found: Requested metadata element is not present --------------------------------
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 09:46 An: Sven Achtelik; Martin Sivak Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Yedidyah Bar David" <didi@redhat.com> Cc: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org Sent: Monday, April 27, 2015 10:34:13 AM Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Hi Did,
results are --------------------------- [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1 [root@ovirt-node2 ~]# echo test > /dev/stdout test --------------------------- Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
Sven
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 08:57 An: Sven Achtelik Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
Best, -- Didi
----- Original Message -----
From: "Martin Sivak" <msivak@redhat.com> To: "Sven Achtelik" <Sven.Achtelik@mailpool.us> Cc: "Yedidyah Bar David" <didi@redhat.com>, "Roy Golan" <rgolan@redhat.com>, users@ovirt.org Sent: Monday, April 27, 2015 11:50:30 AM Subject: Re: AW: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Uh this really is weird.
The situation is clear though:
Broker dies when it tries to initialize logging (missing /dev/stdout ???) Agent dies because it can't connect to the broker.
My /dev/stdout looks like this:
lrwxrwxrwx. 1 root root 15 Mar 30 17:29 /dev/stdout -> /proc/self/fd/1
And /proc/self/fd/1 is obviously related to the process. But I have an idea.
Can you check whether the /proc/self/fd/1 is there? It might be missing if the broker closed its stdout during daemonizing.
If that's the problem, he can't see that - in his shell, /proc/self is its shell's process. Not sure how else to check without some small patch...
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Yes,
----------------------- [root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl start ovirt-ha-agent.service Job for ovirt-ha-broker.service failed. See 'systemctl status ovirt-ha-broker.service' and 'journalctl -xn' for details. [root@ovirt-node2 ~]# journalctl -xn -- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27 02:49:33 CDT. -- Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: stream = open(self.baseFilename, self.mode) Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: [FAILED] Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. -- Subject: Unit ovirt-ha-broker.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ovirt-ha-broker.service has failed. -- -- The result is failed. Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to broker, the number of errors has exceeded the limit ( Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR failed to retrieve Hosted Engine HA info Traceback (most recent call last): File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo stats = instance.get_all_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 97, in get_all_stats with broker.connection(): File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 99, in connection self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 78, in connect raise BrokerConnectionError(error_msg) BrokerConnectionError: Failed to connect to broker, the number of errors has exceeded the limit (5) Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not found: Requested metadata element is not present --------------------------------
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 09:46 An: Sven Achtelik; Martin Sivak Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Yedidyah Bar David" <didi@redhat.com> Cc: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org Sent: Monday, April 27, 2015 10:34:13 AM Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Hi Did,
results are --------------------------- [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1 [root@ovirt-node2 ~]# echo test > /dev/stdout test --------------------------- Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
Sven
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 08:57 An: Sven Achtelik Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
Best, -- Didi
-- Didi
Hi Martin, on my root login things look like this: [root@ovirt-node2 ~]# ls -lah /proc/self/fd/1 lrwx------ 1 root root 64 Apr 27 04:02 /proc/self/fd/1 -> /dev/pts/1 Sven -----Ursprüngliche Nachricht----- Von: Martin Sivak [mailto:msivak@redhat.com] Gesendet: Montag, 27. April 2015 10:51 An: Sven Achtelik Cc: Yedidyah Bar David; Roy Golan; users@ovirt.org Betreff: Re: AW: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host Uh this really is weird. The situation is clear though: Broker dies when it tries to initialize logging (missing /dev/stdout ???) Agent dies because it can't connect to the broker. My /dev/stdout looks like this: lrwxrwxrwx. 1 root root 15 Mar 30 17:29 /dev/stdout -> /proc/self/fd/1 And /proc/self/fd/1 is obviously related to the process. But I have an idea. Can you check whether the /proc/self/fd/1 is there? It might be missing if the broker closed its stdout during daemonizing. -- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message -----
Yes,
----------------------- [root@ovirt-node2 ~]# systemctl start ovirt-ha-broker.service && systemctl start ovirt-ha-agent.service Job for ovirt-ha-broker.service failed. See 'systemctl status ovirt-ha-broker.service' and 'journalctl -xn' for details. [root@ovirt-node2 ~]# journalctl -xn -- Logs begin at Sun 2015-04-26 09:14:15 CDT, end at Mon 2015-04-27 02:49:33 CDT. -- Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: stream = open(self.baseFilename, self.mode) Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[29068]: [FAILED] Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. -- Subject: Unit ovirt-ha-broker.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ovirt-ha-broker.service has failed. -- -- The result is failed. Apr 27 02:49:27 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to broker, the number of errors has exceeded the limit ( Apr 27 02:49:33 ovirt-node2.mgmt.asl.local vdsm[3309]: vdsm vds ERROR failed to retrieve Hosted Engine HA info Traceback (most recent call last): File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo stats = instance.get_all_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 97, in get_all_stats with broker.connection(): File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 99, in connection self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 78, in connect raise BrokerConnectionError(error_msg) BrokerConnectionError: Failed to connect to broker, the number of errors has exceeded the limit (5) Apr 27 02:49:33 ovirt-node2.mgmt.asl.local libvirtd[1678]: metadata not found: Requested metadata element is not present --------------------------------
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 09:46 An: Sven Achtelik; Martin Sivak Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Yedidyah Bar David" <didi@redhat.com> Cc: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org Sent: Monday, April 27, 2015 10:34:13 AM Subject: AW: AW: [ovirt-users] Hosted Engine-Setup issue additional host
Hi Did,
results are --------------------------- [root@ovirt-node2 ~]# ls -l /dev/stdout lrwxrwxrwx 1 root root 15 Apr 26 09:14 /dev/stdout -> /proc/self/fd/1 [root@ovirt-node2 ~]# echo test > /dev/stdout test --------------------------- Looks like everything is working fine.
And it still fails with the same message when you restart ha daemons?
Adding Martin.
Weird.
Sven
-----Ursprüngliche Nachricht----- Von: Yedidyah Bar David [mailto:didi@redhat.com] Gesendet: Montag, 27. April 2015 08:57 An: Sven Achtelik Cc: Roy Golan; users@ovirt.org Betreff: Re: AW: [ovirt-users] Hosted Engine-Setup issue additional host
----- Original Message -----
From: "Sven Achtelik" <Sven.Achtelik@mailpool.us> To: "Roy Golan" <rgolan@redhat.com>, users@ovirt.org, "Yedidyah Bar David" <didi@redhat.com> Sent: Sunday, April 26, 2015 6:57:06 PM Subject: AW: [ovirt-users] Hosted Engine-Setup issue additional host
On the node that fails to start the ha-broker and ha-agent I'm using:
ovirt-engine-sdk-python.noarch 3.5.2.1-1.el7.centos @ovirt-3.5-pre ovirt-host-deploy.noarch 1.3.1-1.el7 @ovirt-3.5 ovirt-hosted-engine-ha.noarch 1.2.5-1.el7.centos @ovirt-3.5 ovirt-hosted-engine-setup.noarch 1.2.3-1.el7.centos @ovirt-3.5-pre ovirt-release35.noarch 003-1 @/ovirt-release35
Von: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] Im Auftrag von Roy Golan Gesendet: Sonntag, 26. April 2015 16:59 An: users@ovirt.org; Yedidyah Bar David Betreff: Re: [ovirt-users] Hosted Engine-Setup issue additional host
On 04/26/2015 05:38 PM, Sven Achtelik wrote: Hi All,
after a successful setup of hosted-engine on the first node I'm having trouble completing it on an additional node. The Setup fails with: --------------------------------- [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] Still waiting for VDSM host to become operational... [ INFO ] The VDSM Host is now operational [ INFO ] Enabling and starting HA services [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150426080028.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination --------------------------------- After that the node is added to the cluster and is operational from the GUI, but the hosted engine broker and agent fail to start with error messages: ---------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-agent.service -l ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 20min ago Process: 5373 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-agent start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: IOError: [Errno 6] No such device or address: '/dev/stdout' Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-agent[5373]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-agent.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Monitoring Agent. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-agent.service entered failed state. ------------------------------------- And ------------------------------------- [root@ovirt-node2 ~]# systemctl status ovirt-ha-broker ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled) Active: failed (Result: exit-code) since Sun 2015-04-26 08:00:28 CDT; 21min ago Process: 5359 ExecStart=/usr/lib/systemd/systemd-ovirt-ha-broker start (code=exited, status=1/FAILURE)
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: hdlr = FileHandler(filename, mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...it__ Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: StreamHandler.__init__(self, self._open()) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: File "/usr/lib64/python2.7/logging/__init__.py", line ...open Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: stream = open(self.baseFilename, self.mode) Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: IOError: [Errno 6] No such device or address: '/dev/stdout'
Didi any clue? the log says it runs as root so I canrule that out
That's weird. Please check/post:
ls -l /dev/stdout echo test > /dev/stdout
It should be a symlink to /proc/self/fd/1 .
Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd-ovirt-ha-broker[5359]: [FAILED] Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: ovirt-ha-broker.service: control process exited, code=exited status=1 Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Failed to start oVirt Hosted Engine High Availability Communications Broker. Apr 26 08:00:28 ovirt-node2.mgmt.asl.local systemd[1]: Unit ovirt-ha-broker.service entered failed state. ----------------------------------------
The system is a CentOS 7 Setup with SeLinux switched off, no firewall or iptables. How can I find out which version of ovirt I'm running exactly ? I've had a lock at the logs and read through old bug reports.
the rpm version of ovirt* will be enough I guess
Thank you,
Sven
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
Thanks, -- Didi
Best, -- Didi
participants (4)
-
Martin Sivak -
Roy Golan -
Sven Achtelik -
Yedidyah Bar David