It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade?

Can you check when were the services started and compare it? ps -eo pid,comm,lstart,etime,time,args

-- Martin Sivák msivak(a)redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message ----- > Hello, > > might this be a bug? Logs are filling up with his on hosts the engine is > *not* running on. When I migrate the engine to this host, the error > stops and starts on the host migrated from. If so, I would open a bug. > > CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 > x86_64 x86_64 x86_64 GNU/Linux > ovirt-hosted-engine-ha-1.2.4-1.el6.noarch > > example > -------------------------- > Oct 22 00:44:15 nodehv02 ovirt-ha-broker > ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error > handling request, data: 'get-stats > storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent > service_type=hosted-engine'#012Traceback (most recent call last):#012 > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", > line 166, in handle#012 data)#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", > line 268, in _dispatch#012 .get_all_stats_for_service_type(client, > **options)#012TypeError: get_all_stats_for_service_type() got an > unexpected keyword argument 'storage_dir' > Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine > HA info#012Traceback (most recent call last):#012 File > "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = > instance.get_all_stats()#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", > line 86, in get_all_stats#012 > self._check_liveness_metadata(stats[host_id], broker)#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", > line 171, in get_stats_from_storage#012 try:#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", > line 199, in _checked_communicate#012 """#012RequestError: Request > failed: <type 'exceptions.TypeError'> > ------------------------- > > -- > Daniel Helgenberger > m box bewegtbild GmbH > > P: +49/30/2408781-22 > F: +49/30/2408781-10 > > ACKERSTR. 19 > D-10115 BERLIN > > > www.m-box.de www.monkeymen.tv > > Geschäftsführer: Martin Retschitzegger / Michaela Göllner > Handeslregister: Amtsgericht Charlottenburg / HRB 112767 > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users >

Ah, what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin. -- Martin Sivák msivak(a)redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message ----- > On 23.10.2014 11:09, Martin Sivak wrote: >> It seems that your running version of broker is not compatible with the >> agent. Did you restart both services after upgrade? > I did, please see below. >> Can you check when were the services started and compare it? >> >> ps -eo pid,comm,lstart,etime,time,args > 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 > /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent > 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 > /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker > >> -- >> Martin Sivák >> msivak(a)redhat.com >> Red Hat Czech >> RHEV-M SLA / Brno, CZ >> >> ----- Original Message ----- >>> Hello, >>> >>> might this be a bug? Logs are filling up with his on hosts the engine is >>> *not* running on. When I migrate the engine to this host, the error >>> stops and starts on the host migrated from. If so, I would open a bug. >>> >>> CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 >>> x86_64 x86_64 x86_64 GNU/Linux >>> ovirt-hosted-engine-ha-1.2.4-1.el6.noarch >>> >>> example >>> -------------------------- >>> Oct 22 00:44:15 nodehv02 ovirt-ha-broker >>> ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error >>> handling request, data: 'get-stats >>> storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent >>> service_type=hosted-engine'#012Traceback (most recent call last):#012 >>> File >>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >>> line 166, in handle#012 data)#012 File >>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >>> line 268, in _dispatch#012 .get_all_stats_for_service_type(client, >>> **options)#012TypeError: get_all_stats_for_service_type() got an >>> unexpected keyword argument 'storage_dir' >>> Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine >>> HA info#012Traceback (most recent call last):#012 File >>> "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = >>> instance.get_all_stats()#012 File >>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", >>> line 86, in get_all_stats#012 >>> self._check_liveness_metadata(stats[host_id], broker)#012 File >>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >>> line 171, in get_stats_from_storage#012 try:#012 File >>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >>> line 199, in _checked_communicate#012 """#012RequestError: Request >>> failed: <type 'exceptions.TypeError'> >>> ------------------------- >>> >>> -- >>> Daniel Helgenberger >>> m box bewegtbild GmbH >>> >>> P: +49/30/2408781-22 >>> F: +49/30/2408781-10 >>> >>> ACKERSTR. 19 >>> D-10115 BERLIN >>> >>> >>> www.m-box.de www.monkeymen.tv >>> >>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >>> > -- > Daniel Helgenberger > m box bewegtbild GmbH > > P: +49/30/2408781-22 > F: +49/30/2408781-10 > > ACKERSTR. 19 > D-10115 BERLIN > > > www.m-box.de www.monkeymen.tv > > Geschäftsführer: Martin Retschitzegger / Michaela Göllner > Handeslregister: Amtsgericht Charlottenburg / HRB 112767 > >

Hi, yes it is possible. VDSM is probably still using the old version of the brokerlink library because of it.

-- Martin Sivák msivak(a)redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message ----- > Hm, > > I upgraded vdsm and restarted it *before* iI upgraded engine HA; > 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 > /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid > > Might this be the problem? > > On 23.10.2014 12:12, Martin Sivak wrote: >> Ah, >> >> what about vdsm? It queries the broker every now and then to get the >> information it reports to the webadmin. >> >> -- >> Martin Sivák >> msivak(a)redhat.com >> Red Hat Czech >> RHEV-M SLA / Brno, CZ >> >> ----- Original Message ----- >>> On 23.10.2014 11:09, Martin Sivak wrote: >>>> It seems that your running version of broker is not compatible with the >>>> agent. Did you restart both services after upgrade? >>> I did, please see below. >>>> Can you check when were the services started and compare it? >>>> >>>> ps -eo pid,comm,lstart,etime,time,args >>> 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 >>> /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent >>> 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 >>> /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker >>> >>>> -- >>>> Martin Sivák >>>> msivak(a)redhat.com >>>> Red Hat Czech >>>> RHEV-M SLA / Brno, CZ >>>> >>>> ----- Original Message ----- >>>>> Hello, >>>>> >>>>> might this be a bug? Logs are filling up with his on hosts the engine is >>>>> *not* running on. When I migrate the engine to this host, the error >>>>> stops and starts on the host migrated from. If so, I would open a bug. >>>>> >>>>> CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 >>>>> x86_64 x86_64 x86_64 GNU/Linux >>>>> ovirt-hosted-engine-ha-1.2.4-1.el6.noarch >>>>> >>>>> example >>>>> -------------------------- >>>>> Oct 22 00:44:15 nodehv02 ovirt-ha-broker >>>>> ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error >>>>> handling request, data: 'get-stats >>>>> storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent >>>>> service_type=hosted-engine'#012Traceback (most recent call last):#012 >>>>> File >>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >>>>> line 166, in handle#012 data)#012 File >>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >>>>> line 268, in _dispatch#012 .get_all_stats_for_service_type(client, >>>>> **options)#012TypeError: get_all_stats_for_service_type() got an >>>>> unexpected keyword argument 'storage_dir' >>>>> Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine >>>>> HA info#012Traceback (most recent call last):#012 File >>>>> "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = >>>>> instance.get_all_stats()#012 File >>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", >>>>> line 86, in get_all_stats#012 >>>>> self._check_liveness_metadata(stats[host_id], broker)#012 File >>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >>>>> line 171, in get_stats_from_storage#012 try:#012 File >>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >>>>> line 199, in _checked_communicate#012 """#012RequestError: Request >>>>> failed: <type 'exceptions.TypeError'> >>>>> ------------------------- >>>>> >>>>> -- >>>>> Daniel Helgenberger >>>>> m box bewegtbild GmbH >>>>> >>>>> P: +49/30/2408781-22 >>>>> F: +49/30/2408781-10 >>>>> >>>>> ACKERSTR. 19 >>>>> D-10115 BERLIN >>>>> >>>>> >>>>> www.m-box.de www.monkeymen.tv >>>>> >>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>> >>> -- >>> Daniel Helgenberger >>> m box bewegtbild GmbH >>> >>> P: +49/30/2408781-22 >>> F: +49/30/2408781-10 >>> >>> ACKERSTR. 19 >>> D-10115 BERLIN >>> >>> >>> www.m-box.de www.monkeymen.tv >>> >>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>> >>> > -- > Daniel Helgenberger > m box bewegtbild GmbH > > P: +49/30/2408781-22 > F: +49/30/2408781-10 > > ACKERSTR. 19 > D-10115 BERLIN > > > www.m-box.de www.monkeymen.tv > > Geschäftsführer: Martin Retschitzegger / Michaela Göllner > Handeslregister: Amtsgericht Charlottenburg / HRB 112767 > >

Il 23/10/2014 15:30, Daniel Helgenberger ha scritto: > On 23.10.2014 14:15, Martin Sivak wrote: >> Hi, >> >> yes it is possible. VDSM is probably still using the old version of the brokerlink library because of it. > thanks Martin, this seems to have worked. > Question: is VDSM fully backward compatible with ovirt-engine-ha? > > Anyway, my conclusion is *not* to use host-deploy any more on HA nodes > but rather yum upadate/manual service restart. Also, it might be a good > idea for host-deploy to detect ovirt-engine-ha and upgrade it along with > VDSM if applicable. If you or someone agrees I will file an RFE .. > though Allon might not be happy ;) +1 for 3.6 host-deploy must be aware of hosted engine anyway for allowing to add new host to the HE cluster from the web admin interface.

>> >> >> -- >> Martin Sivák >> msivak(a)redhat.com >> Red Hat Czech >> RHEV-M SLA / Brno, CZ >> >> ----- Original Message ----- >>> Hm, >>> >>> I upgraded vdsm and restarted it *before* iI upgraded engine HA; >>> 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 >>> /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid >>> >>> Might this be the problem? >>> >>> On 23.10.2014 12:12, Martin Sivak wrote: >>>> Ah, >>>> >>>> what about vdsm? It queries the broker every now and then to get the >>>> information it reports to the webadmin. >>>> >>>> -- >>>> Martin Sivák >>>> msivak(a)redhat.com >>>> Red Hat Czech >>>> RHEV-M SLA / Brno, CZ >>>> >>>> ----- Original Message ----- >>>>> On 23.10.2014 11:09, Martin Sivak wrote: >>>>>> It seems that your running version of broker is not compatible with the >>>>>> agent. Did you restart both services after upgrade? >>>>> I did, please see below. >>>>>> Can you check when were the services started and compare it? >>>>>> >>>>>> ps -eo pid,comm,lstart,etime,time,args >>>>> 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 >>>>> /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent >>>>> 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 >>>>> /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker >>>>> >>>>>> -- >>>>>> Martin Sivák >>>>>> msivak(a)redhat.com >>>>>> Red Hat Czech >>>>>> RHEV-M SLA / Brno, CZ >>>>>> >>>>>> ----- Original Message ----- >>>>>>> Hello, >>>>>>> >>>>>>> might this be a bug? Logs are filling up with his on hosts the engine is >>>>>>> *not* running on. When I migrate the engine to this host, the error >>>>>>> stops and starts on the host migrated from. If so, I would open a bug. >>>>>>> >>>>>>> CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 >>>>>>> x86_64 x86_64 x86_64 GNU/Linux >>>>>>> ovirt-hosted-engine-ha-1.2.4-1.el6.noarch >>>>>>> >>>>>>> example >>>>>>> -------------------------- >>>>>>> Oct 22 00:44:15 nodehv02 ovirt-ha-broker >>>>>>> ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error >>>>>>> handling request, data: 'get-stats >>>>>>> storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent >>>>>>> service_type=hosted-engine'#012Traceback (most recent call last):#012 >>>>>>> File >>>>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >>>>>>> line 166, in handle#012 data)#012 File >>>>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >>>>>>> line 268, in _dispatch#012 .get_all_stats_for_service_type(client, >>>>>>> **options)#012TypeError: get_all_stats_for_service_type() got an >>>>>>> unexpected keyword argument 'storage_dir' >>>>>>> Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine >>>>>>> HA info#012Traceback (most recent call last):#012 File >>>>>>> "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = >>>>>>> instance.get_all_stats()#012 File >>>>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", >>>>>>> line 86, in get_all_stats#012 >>>>>>> self._check_liveness_metadata(stats[host_id], broker)#012 File >>>>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >>>>>>> line 171, in get_stats_from_storage#012 try:#012 File >>>>>>> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >>>>>>> line 199, in _checked_communicate#012 """#012RequestError: Request >>>>>>> failed: <type 'exceptions.TypeError'> >>>>>>> ------------------------- >>>>>>> >>>>>>> -- >>>>>>> Daniel Helgenberger >>>>>>> m box bewegtbild GmbH >>>>>>> >>>>>>> P: +49/30/2408781-22 >>>>>>> F: +49/30/2408781-10 >>>>>>> >>>>>>> ACKERSTR. 19 >>>>>>> D-10115 BERLIN >>>>>>> >>>>>>> >>>>>>> www.m-box.de www.monkeymen.tv >>>>>>> >>>>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Users mailing list >>>>>>> Users(a)ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>> >>>>> -- >>>>> Daniel Helgenberger >>>>> m box bewegtbild GmbH >>>>> >>>>> P: +49/30/2408781-22 >>>>> F: +49/30/2408781-10 >>>>> >>>>> ACKERSTR. 19 >>>>> D-10115 BERLIN >>>>> >>>>> >>>>> www.m-box.de www.monkeymen.tv >>>>> >>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>>>> >>>>> >>> -- >>> Daniel Helgenberger >>> m box bewegtbild GmbH >>> >>> P: +49/30/2408781-22 >>> F: +49/30/2408781-10 >>> >>> ACKERSTR. 19 >>> D-10115 BERLIN >>> >>> >>> www.m-box.de www.monkeymen.tv >>> >>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>> >>>