ovirt-engine-ha: 012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir'
Hello, might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug. CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> ------------------------- -- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? Can you check when were the services started and compare it? ps -eo pid,comm,lstart,etime,time,args -- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message -----
Hello,
might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug.
CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch
example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> -------------------------
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 23.10.2014 11:09, Martin Sivak wrote:
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? I did, please see below. Can you check when were the services started and compare it?
ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hello,
might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug.
CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch
example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> -------------------------
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Ah, what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin. -- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message -----
On 23.10.2014 11:09, Martin Sivak wrote:
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? I did, please see below. Can you check when were the services started and compare it?
ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hello,
might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug.
CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch
example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> -------------------------
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Hm, I upgraded vdsm and restarted it *before* iI upgraded engine HA; 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid Might this be the problem? On 23.10.2014 12:12, Martin Sivak wrote:
Ah,
what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin.
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
On 23.10.2014 11:09, Martin Sivak wrote:
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? I did, please see below. Can you check when were the services started and compare it?
ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hello,
might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug.
CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch
example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> -------------------------
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Hi, yes it is possible. VDSM is probably still using the old version of the brokerlink library because of it. -- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ ----- Original Message -----
Hm,
I upgraded vdsm and restarted it *before* iI upgraded engine HA; 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid
Might this be the problem?
On 23.10.2014 12:12, Martin Sivak wrote:
Ah,
what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin.
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
On 23.10.2014 11:09, Martin Sivak wrote:
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? I did, please see below. Can you check when were the services started and compare it?
ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hello,
might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug.
CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch
example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> -------------------------
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Hi,
yes it is possible. VDSM is probably still using the old version of the brokerlink library because of it.
On 23.10.2014 14:15, Martin Sivak wrote: thanks Martin, this seems to have worked. Question: is VDSM fully backward compatible with ovirt-engine-ha? Anyway, my conclusion is *not* to use host-deploy any more on HA nodes but rather yum upadate/manual service restart. Also, it might be a good idea for host-deploy to detect ovirt-engine-ha and upgrade it along with VDSM if applicable. If you or someone agrees I will file an RFE .. though Allon might not be happy ;)
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hm,
I upgraded vdsm and restarted it *before* iI upgraded engine HA; 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid
Might this be the problem?
On 23.10.2014 12:12, Martin Sivak wrote:
Ah,
what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin.
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
On 23.10.2014 11:09, Martin Sivak wrote:
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? I did, please see below. Can you check when were the services started and compare it?
ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hello,
might this be a bug? Logs are filling up with his on hosts the engine is *not* running on. When I migrate the engine to this host, the error stops and starts on the host migrated from. If so, I would open a bug.
CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ovirt-hosted-engine-ha-1.2.4-1.el6.noarch
example -------------------------- Oct 22 00:44:15 nodehv02 ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'get-stats storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent service_type=hosted-engine'#012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 268, in _dispatch#012 .get_all_stats_for_service_type(client, **options)#012TypeError: get_all_stats_for_service_type() got an unexpected keyword argument 'storage_dir' Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 86, in get_all_stats#012 self._check_liveness_metadata(stats[host_id], broker)#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_storage#012 try:#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 199, in _checked_communicate#012 """#012RequestError: Request failed: <type 'exceptions.TypeError'> -------------------------
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
Il 23/10/2014 15:30, Daniel Helgenberger ha scritto:
Hi,
yes it is possible. VDSM is probably still using the old version of the brokerlink library because of it.
On 23.10.2014 14:15, Martin Sivak wrote: thanks Martin, this seems to have worked. Question: is VDSM fully backward compatible with ovirt-engine-ha?
Anyway, my conclusion is *not* to use host-deploy any more on HA nodes but rather yum upadate/manual service restart. Also, it might be a good idea for host-deploy to detect ovirt-engine-ha and upgrade it along with VDSM if applicable. If you or someone agrees I will file an RFE .. though Allon might not be happy ;)
+1 for 3.6 host-deploy must be aware of hosted engine anyway for allowing to add new host to the HE cluster from the web admin interface.
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hm,
I upgraded vdsm and restarted it *before* iI upgraded engine HA; 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid
Might this be the problem?
On 23.10.2014 12:12, Martin Sivak wrote:
Ah,
what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin.
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
On 23.10.2014 11:09, Martin Sivak wrote:
It seems that your running version of broker is not compatible with the agent. Did you restart both services after upgrade? I did, please see below. Can you check when were the services started and compare it?
ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message ----- > Hello, > > might this be a bug? Logs are filling up with his on hosts the engine is > *not* running on. When I migrate the engine to this host, the error > stops and starts on the host migrated from. If so, I would open a bug. > > CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 > x86_64 x86_64 x86_64 GNU/Linux > ovirt-hosted-engine-ha-1.2.4-1.el6.noarch > > example > -------------------------- > Oct 22 00:44:15 nodehv02 ovirt-ha-broker > ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error > handling request, data: 'get-stats > storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent > service_type=hosted-engine'#012Traceback (most recent call last):#012 > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", > line 166, in handle#012 data)#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", > line 268, in _dispatch#012 .get_all_stats_for_service_type(client, > **options)#012TypeError: get_all_stats_for_service_type() got an > unexpected keyword argument 'storage_dir' > Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine > HA info#012Traceback (most recent call last):#012 File > "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = > instance.get_all_stats()#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", > line 86, in get_all_stats#012 > self._check_liveness_metadata(stats[host_id], broker)#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", > line 171, in get_stats_from_storage#012 try:#012 File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", > line 199, in _checked_communicate#012 """#012RequestError: Request > failed: <type 'exceptions.TypeError'> > ------------------------- > > -- > Daniel Helgenberger > m box bewegtbild GmbH > > P: +49/30/2408781-22 > F: +49/30/2408781-10 > > ACKERSTR. 19 > D-10115 BERLIN > > > www.m-box.de www.monkeymen.tv > > Geschäftsführer: Martin Retschitzegger / Michaela Göllner > Handeslregister: Amtsgericht Charlottenburg / HRB 112767 > > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > -- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
On 23.10.2014 15:39, Sandro Bonazzola wrote:
Il 23/10/2014 15:30, Daniel Helgenberger ha scritto:
Hi,
yes it is possible. VDSM is probably still using the old version of the brokerlink library because of it.
On 23.10.2014 14:15, Martin Sivak wrote: thanks Martin, this seems to have worked. Question: is VDSM fully backward compatible with ovirt-engine-ha?
Anyway, my conclusion is *not* to use host-deploy any more on HA nodes but rather yum upadate/manual service restart. Also, it might be a good idea for host-deploy to detect ovirt-engine-ha and upgrade it along with VDSM if applicable. If you or someone agrees I will file an RFE .. though Allon might not be happy ;) +1 for 3.6 host-deploy must be aware of hosted engine anyway for allowing to add new host to the HE cluster from the web admin interface. Added, https://bugzilla.redhat.com/show_bug.cgi?id=1156163
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
Hm,
I upgraded vdsm and restarted it *before* iI upgraded engine HA; 7042 vdsm Sun Oct 19 16:24:24 2014 3-19:05:00 18:20:44 /usr/bin/python /usr/share/vdsm/vdsm --pidfile /var/run/vdsm/vdsmd.pid
Might this be the problem?
On 23.10.2014 12:12, Martin Sivak wrote:
Ah,
what about vdsm? It queries the broker every now and then to get the information it reports to the webadmin.
-- Martin Sivák msivak@redhat.com Red Hat Czech RHEV-M SLA / Brno, CZ
----- Original Message -----
On 23.10.2014 11:09, Martin Sivak wrote: > It seems that your running version of broker is not compatible with the > agent. Did you restart both services after upgrade? I did, please see below. > Can you check when were the services started and compare it? > > ps -eo pid,comm,lstart,etime,time,args 15474 ovirt-ha-agent Tue Oct 21 13:39:56 2014 1-21:49:28 00:09:38 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent 15568 ovirt-ha-broker Tue Oct 21 13:40:06 2014 1-21:49:18 00:54:01 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker
> -- > Martin Sivák > msivak@redhat.com > Red Hat Czech > RHEV-M SLA / Brno, CZ > > ----- Original Message ----- >> Hello, >> >> might this be a bug? Logs are filling up with his on hosts the engine is >> *not* running on. When I migrate the engine to this host, the error >> stops and starts on the host migrated from. If so, I would open a bug. >> >> CentOS 2.6.32-431.29.2.el6.x86_64 #1 SMP Tue Sep 9 21:36:05 UTC 2014 >> x86_64 x86_64 x86_64 GNU/Linux >> ovirt-hosted-engine-ha-1.2.4-1.el6.noarch >> >> example >> -------------------------- >> Oct 22 00:44:15 nodehv02 ovirt-ha-broker >> ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error >> handling request, data: 'get-stats >> storage_dir=/rhev/data-center/mnt/192.168.50.11:_volumes_tank_ovirt-vdsmd/bcfa7ec4-5278-44d8-9f31-682f2d9de91d/ha_agent >> service_type=hosted-engine'#012Traceback (most recent call last):#012 >> File >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >> line 166, in handle#012 data)#012 File >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/broker/listener.py", >> line 268, in _dispatch#012 .get_all_stats_for_service_type(client, >> **options)#012TypeError: get_all_stats_for_service_type() got an >> unexpected keyword argument 'storage_dir' >> Oct 22 00:44:15 nodehv02 vdsm vds ERROR failed to retrieve Hosted Engine >> HA info#012Traceback (most recent call last):#012 File >> "/usr/share/vdsm/API.py", line 1678, in _getHaInfo#012 stats = >> instance.get_all_stats()#012 File >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", >> line 86, in get_all_stats#012 >> self._check_liveness_metadata(stats[host_id], broker)#012 File >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >> line 171, in get_stats_from_storage#012 try:#012 File >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", >> line 199, in _checked_communicate#012 """#012RequestError: Request >> failed: <type 'exceptions.TypeError'> >> ------------------------- >> >> -- >> Daniel Helgenberger >> m box bewegtbild GmbH >> >> P: +49/30/2408781-22 >> F: +49/30/2408781-10 >> >> ACKERSTR. 19 >> D-10115 BERLIN >> >> >> www.m-box.de www.monkeymen.tv >> >> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >> >> _______________________________________________ >> Users mailing list >> Users@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> -- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH
P: +49/30/2408781-22 F: +49/30/2408781-10
ACKERSTR. 19 D-10115 BERLIN
www.m-box.de www.monkeymen.tv
Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
-- Daniel Helgenberger m box bewegtbild GmbH P: +49/30/2408781-22 F: +49/30/2408781-10 ACKERSTR. 19 D-10115 BERLIN www.m-box.de www.monkeymen.tv Geschäftsführer: Martin Retschitzegger / Michaela Göllner Handeslregister: Amtsgericht Charlottenburg / HRB 112767
participants (3)
-
Daniel Helgenberger -
Martin Sivak -
Sandro Bonazzola