[ovirt-users] oVirt storage access failure from host

[Alex K]

Hi All,

I have a 3 server ovirt 4.1 selft hosted setup with gluster replica 3.

I see that suddenly one of the hosts reported as unresponsive and at same
time the /var/log/messages logged:

ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler
ERROR Error handling request, data: 'set-storage-domain FilesystemBackend
dom_type=glusterfs
sd_uuid=ad7b9e2a-7ae3-46ad-9429-5f5ef452eac8'#012Traceback (most recent
call last):#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py",
line 166, in handle#012    data)#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py",
line 299, in _dispatch#012    .set_storage_domain(client, sd_type,
**options)#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py",
line 66, in set_storage_domain#012    self._backends[client].connect()#012
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py",
line 462, in connect#012    self._dom_type)#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py",
line 107, in get_domain_path#012    " in {1}".format(sd_uuid,
parent))#012BackendFailureException: path to storage domain
ad7b9e2a-7ae3-46ad-9429-5f5ef452eac8 not found in
/rhev/data-center/mnt/glusterSD
Jan 15 11:04:56 v1 journal: vdsm root ERROR failed to retrieve Hosted
Engine HA info#012Traceback (most recent call last):#012  File
"/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in
_getHaInfo#012    stats = instance.get_all_stats()#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
line 103, in get_all_stats#012    self._configure_broker_conn(broker)#012
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
line 180, in _configure_broker_conn#012    dom_type=dom_type)#012  File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 177, in set_storage_domain#012    .format(sd_type, options,
e))#012RequestError: Failed to set storage domain FilesystemBackend,
options {'dom_type': 'glusterfs', 'sd_uuid':
'ad7b9e2a-7ae3-46ad-9429-5f5ef452eac8'}: Request failed: <class
'ovirt_hosted_engine_ha.lib.storage_backends.BackendFailureException'>


At VDSM logs i see the following continuously logged:
[jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00
seconds (__init__:539)

No errors seen at gluster at same time frame.

Any hints on what is causing this issue? It seems a storage access issue
but gluster was up and volumes ok. The VMs that I am running on top are
Windows 10 and Windows 2016 64 bit.


Thanx,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180119/05b58427/attachment.html>