[ovirt-users] oVirt Hosts fail to Upgrade

Markus.Schaufler at ooe.gv.at Markus.Schaufler at ooe.gv.at
Mon Feb 26 14:14:56 UTC 2018 

Hi!

I recently installed a 4-node Test-Cluster with FC storage for a PoC with a self-hosted-engine and some Test VMs running.
I tried to upgrade the hosts via Webinterface via Maintainance -> Check for Upgrade -> Upgrade.
They fail all with the same errors in the messages log on each host:

Feb 26 15:04:31 VIGT01-101 sanlock[2444]: 2018-02-26 15:04:31 432734 [2460]: s324 add_lockspace fail result -19
Feb 26 15:04:36 VIGT01-101 journal: ovirt-ha-broker ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker ERROR Failed to read metadata from /var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/f4ac3b2b-5219-4e22-b6c9-1520ce1ce3e0/0977a863-90b0-4ddb-8559-ed7770c38f69#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 151, in get_raw_stats#012    f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC)#012OSError: [Errno 2] No such file or directory: '/var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/f4ac3b2b-5219-4e22-b6c9-1520ce1ce3e0/0977a863-90b0-4ddb-8559-ed7770c38f69'
Feb 26 15:04:36 VIGT01-101 journal: ovirt-ha-broker ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update ERROR Failed to read state.#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 88, in run#012    self._storage_broker.get_raw_stats()#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 162, in get_raw_stats#012    .format(str(e)))#012RequestError: failed to read metadata: [Errno 2] No such file or directory: '/var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/f4ac3b2b-5219-4e22-b6c9-1520ce1ce3e0/0977a863-90b0-4ddb-8559-ed7770c38f69'
Feb 26 15:04:36 VIGT01-101 journal: ovirt-ha-broker ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update ERROR Failed to update state.#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 77, in run#012    if (self._status_broker._inquire_whiteboard_lock() or#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 183, in _inquire_whiteboard_lock#012    self.host_id, self._lease_file)#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 121, in host_id#012    raise ex.HostIdNotLockedError("Host id is not set")#012HostIdNotLockedError: Host id is not set
Feb 26 15:04:36 VIGT01-101 journal: ovirt-ha-broker ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker ERROR Failed to read metadata from /var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/f4ac3b2b-5219-4e22-b6c9-1520ce1ce3e0/0977a863-90b0-4ddb-8559-ed7770c38f69#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 151, in get_raw_stats#012    f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC)#012OSError: [Errno 2] No such file or directory: '/var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/f4ac3b2b-5219-4e22-b6c9-1520ce1ce3e0/0977a863-90b0-4ddb-8559-ed7770c38f69'
Feb 26 15:04:36 VIGT01-101 journal: ovirt-ha-broker ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update ERROR Failed to read state.#012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 88, in run#012    self._storage_broker.get_raw_stats()#012  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 162, in get_raw_stats#012    .format(str(e)))#012RequestError: failed to read metadata: [Errno 2] No such file or directory: '/var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/f4ac3b2b-5219-4e22-b6c9-1520ce1ce3e0/0977a863-90b0-4ddb-8559-ed7770c38f69'
Feb 26 15:04:36 VIGT01-101 sanlock[2444]: 2018-02-26 15:04:36 432739 [422733]: open error -2 /var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/4ba684c0-3d8d-4148-99cd-4f4c50a4714a/3f764582-cb12-4c4e-91d3-8bc13a5427d6
Feb 26 15:04:36 VIGT01-101 sanlock[2444]: 2018-02-26 15:04:36 432739 [422733]: s325 open_disk /var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/4ba684c0-3d8d-4148-99cd-4f4c50a4714a/3f764582-cb12-4c4e-91d3-8bc13a5427d6 error -2
Feb 26 15:04:37 VIGT01-101 sanlock[2444]: 2018-02-26 15:04:37 432740 [6221]: s325 add_lockspace fail result -19
Feb 26 15:04:42 VIGT01-101 sanlock[2444]: 2018-02-26 15:04:42 432745 [422771]: open error -2 /var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/4ba684c0-3d8d-4148-99cd-4f4c50a4714a/3f764582-cb12-4c4e-91d3-8bc13a5427d6
Feb 26 15:04:42 VIGT01-101 sanlock[2444]: 2018-02-26 15:04:42 432745 [422771]: s326 open_disk /var/run/vdsm/storage/a4647768-2c33-4318-9111-ca995fcc03c8/4ba684c0-3d8d-4148-99cd-4f4c50a4714a/3f764582-cb12-4c4e-91d3-8bc13a5427d6 error -2

I've also attached some logfiles of one Host.

Any thoughts on this?


Markus Schaufler, MSc

Amt der Oö. Landesregierung
Direktion Präsidium
Abteilung Informationstechnologie
Referat ST3 Server
A-4021 Linz, Kärntnerstraße 16
Tel.:        +43 (0)732 7720 - 13138
Fax:        +43 (0)732 7720 - 213255
email:     markus.schaufler at ooe.gv.at<mailto:markus.schaufler at ooe.gv.at>
Internet: www.land-oberoesterreich.gv.at<http://www.land-oberoesterreich.gv.at/>
DVR:       0069264

Der Austausch von Nachrichten mit o.a. Absender via e-mail dient ausschließlich Informationszwecken.
Rechtsgültige Erklärungen dürfen über dieses Medium nur an das offizielle Postfach it.post at ooe.gv.at<mailto:it.post at ooe.gv.at> übermittelt werden.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180226/9c2b894c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logfiles.tar.gz
Type: application/x-gzip
Size: 1815317 bytes
Desc: logfiles.tar.gz
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180226/9c2b894c/attachment.gz>

More information about the Users mailing list