I have one host which has been newly deployed having issues with: [root@host-01 ~]# tail -f /var/log/ovirt-hosted-engine-ha/agent.log MainThread::INFO::2016-08-23 02:39:16,799::hosted_engine::860::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-08-23 02:39:21,834::hosted_engine::860::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-08-23 02:39:26,894::hosted_engine::860::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-08-23 02:39:31,931::hosted_engine::860::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING Now I tried to cleant he metadata, force it, whatever I tried, I get this: [root@host-01 ~]# hosted-engine --clean-metadata --force-cleanup /usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/vds_client.py:25: DeprecationWarning: vdscli uses xmlrpc. since ovirt 3.6 xmlrpc is deprecated, please use vdsm.jsonrpcvdscli from vdsm import vdscli INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha agent 2.0.2 started INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found certificate common name: host-01.my.domain INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Initializing VDSM /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Connecting the storage INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting storage server /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting storage server /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Refreshing the storage domain /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Preparing images INFO:ovirt_hosted_engine_ha.lib.image.Image:Preparing images /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Reloading vm.conf from the shared storage domain INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Trying to get a fresher copy of vm configuration from the OVF_STORE /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:Found OVF_STORE: imgUUID:8bf93e0a-7665-49b1-b183-29d3ecee2fab, volUUID:63ea7e1b-491f-484f-87e3-d0c1c5de8b9b /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:Found OVF_STORE: imgUUID:774d6560-62ec-452a-9397-efeb9acff78c, volUUID:9cc05166-af8d-4c9c-bc30-60c324fd3799 /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pending is deprecated. Use Dispatcher.socket.pending instead. pending = getattr(dispatcher, 'pending', lambda: 0) INFO:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:Extracting Engine VM OVF from the OVF_STORE INFO:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:OVF_STORE volume path: /rhev/data-center/mnt/flr-01.my.domain:_vol_vol0_oVIRT__HE/4093ad17-bef5-4e4b-9a16-259a98e20321/images/774d6560-62ec-452a-9397-efeb9acff78c/9cc05166-af8d-4c9c-bc30-60c324fd3799 INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Found an OVF for HE VM, trying to convert INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Got vm.conf from OVF_STORE INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:VDSM domain monitor status: PENDING What I also have seen is that when you run --host-id on a different host it says it cannot do a thing as that host already holds another lock on a file for that host you run it on. Anyone a clue what happens here ? It can access the hosted engine storage, so I wonder why this one doesn't want to do it's thing as it cannot for becoming a HA node it seems. Ideas are welcome. Thanks! Matt