/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:05,560::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:05,561::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:15,623::states::437::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm is running on host 192.168.50.201 (id 1)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:18,504::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:18,504::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:30,404::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:30,404::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:42,397::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:42,397::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:54,406::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:20:54,406::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:04,444::state_machine::160::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Global metadata: {'maintenance': False}
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:04,445::state_machine::165::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host 192.168.50.201 (id 1): {'extra': 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=160017 (Tue Oct 21 20:20:53 2014)\nhost-id=1\nscore=2400\nmaintenance=False\nstate=EngineUp\n', 'hostname': '192.168.50.201', 'alive': True, 'host-id': 1, 'engine-status': {'health': 'good', 'vm': 'up', 'detail': 'up'}, 'score': 2400, 'maintenance': False, 'host-ts': 160017}
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:04,445::state_machine::168::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Local (id 2): {'engine-health': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'bridge': True, 'mem-free': 9009.0, 'maintenance': False, 'cpu-load': 0.0292, 'gateway': True}
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:06,459::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:06,459::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:16,496::states::437::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm is running on host 192.168.50.201 (id 1)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:19,404::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:21:19,405::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:09,950::states::437::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm is running on host 192.168.50.201 (id 1)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:12,365::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:17,440::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:22,499::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:27,558::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:32,616::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:37,675::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:42,733::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:47,792::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:52,883::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:23:57,942::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:02,999::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:08,059::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:13,117::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:18,198::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:23,255::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:28,314::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:33,398::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:38,457::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:43,515::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:48,574::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:53,632::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:24:58,692::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:03,751::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:08,810::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:13,869::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:18,927::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:23,983::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:29,038::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:34,094::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:39,153::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:44,209::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:49,279::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:25:54,336::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:20,572::hosted_engine::571::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock) Acquired lock on host id 2
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:20,572::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineDown (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:20,572::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:30,606::states::459::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine down and local host has best score (2400), attempting to start engine VM
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:30,613::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=1413915990.61 type=state_transition detail=EngineDown-EngineStart hostname='nodehv02.lab.mbox.loc'
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:30,690::brokerlink::120::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineDown-EngineStart) sent? sent
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:33,520::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineStart (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:33,520::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:33,538::hosted_engine::911::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:33,596::hosted_engine::923::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:33,596::hosted_engine::878::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start`
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:34,043::hosted_engine::884::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) stdout: 
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:34,044::hosted_engine::885::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) stderr: 
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::ERROR::2014-10-21 20:26:34,044::hosted_engine::898::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) Engine VM started on localhost
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:34,052::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=1413915994.05 type=state_transition detail=EngineStart-EngineStarting hostname='nodehv02.lab.mbox.loc'
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:34,103::brokerlink::120::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStart-EngineStarting) sent? sent
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:36,513::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineStarting (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:36,514::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:46,546::states::722::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:46,553::state_decorators::95::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout set to Tue Oct 21 20:36:46 2014 while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineStarting'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineStarting'>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:48,409::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineStarting (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:48,410::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:26:58,442::states::722::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:00,502::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineStarting (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:00,502::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:10,541::states::722::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:12,445::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineStarting (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:12,446::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:22,540::states::722::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:24,378::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineStarting (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:24,378::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:34,423::state_decorators::88::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout cleared while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineStarting'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineUp'>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:34,430::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=1413916054.43 type=state_transition detail=EngineStarting-EngineUp hostname='nodehv02.lab.mbox.loc'
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:34,498::brokerlink::120::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStarting-EngineUp) sent? sent
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:38,481::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:38,481::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:48,697::states::394::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:51,391::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:27:51,391::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:03,375::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:03,375::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:15,438::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:15,438::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:27,600::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:27,601::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:40,587::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:40,587::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:50,618::states::394::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:53,423::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:28:53,423::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:05,392::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:05,392::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:17,538::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:17,539::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:30,491::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:30,491::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:43,359::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:43,360::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:53,393::states::394::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:55,372::hosted_engine::327::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 2400)
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-10-21 20:29:55,372::hosted_engine::332::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host 192.168.50.201 (id: 1, score: 0)
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:20:01,822::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-237967::ERROR::2014-10-21 20:20:02,845::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-237968::ERROR::2014-10-21 20:20:02,868::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-237982::ERROR::2014-10-21 20:20:09,627::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-237984::ERROR::2014-10-21 20:20:12,934::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-237985::ERROR::2014-10-21 20:20:12,956::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:20:17,090::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-237999::ERROR::2014-10-21 20:20:19,725::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238001::ERROR::2014-10-21 20:20:23,020::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238002::ERROR::2014-10-21 20:20:23,041::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238011::ERROR::2014-10-21 20:20:29,813::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:20:32,392::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238019::ERROR::2014-10-21 20:20:33,107::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238020::ERROR::2014-10-21 20:20:33,129::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238025::ERROR::2014-10-21 20:20:39,905::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238035::ERROR::2014-10-21 20:20:43,199::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238036::ERROR::2014-10-21 20:20:43,213::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:20:47,623::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238041::ERROR::2014-10-21 20:20:49,995::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238046::ERROR::2014-10-21 20:20:53,294::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238047::ERROR::2014-10-21 20:20:53,319::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238061::ERROR::2014-10-21 20:21:00,100::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:21:02,813::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238065::ERROR::2014-10-21 20:21:03,523::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238064::ERROR::2014-10-21 20:21:03,525::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238078::ERROR::2014-10-21 20:21:10,289::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238080::ERROR::2014-10-21 20:21:12,749::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238081::ERROR::2014-10-21 20:21:12,772::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:21:17,997::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238095::ERROR::2014-10-21 20:21:20,458::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238097::ERROR::2014-10-21 20:21:22,837::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238098::ERROR::2014-10-21 20:21:22,861::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238105::ERROR::2014-10-21 20:23:09,948::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238107::ERROR::2014-10-21 20:23:09,949::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238108::ERROR::2014-10-21 20:23:09,949::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-221845::ERROR::2014-10-21 20:23:09,950::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238173::ERROR::2014-10-21 20:23:10,413::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238175::ERROR::2014-10-21 20:23:10,445::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238176::ERROR::2014-10-21 20:23:10,447::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238190::ERROR::2014-10-21 20:23:19,613::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238191::ERROR::2014-10-21 20:23:19,678::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238192::ERROR::2014-10-21 20:23:19,704::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238205::ERROR::2014-10-21 20:23:29,827::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238206::ERROR::2014-10-21 20:23:29,892::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238207::ERROR::2014-10-21 20:23:29,921::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238217::ERROR::2014-10-21 20:23:40,073::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238218::ERROR::2014-10-21 20:23:40,145::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238219::ERROR::2014-10-21 20:23:40,159::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238229::ERROR::2014-10-21 20:24:05,302::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238230::ERROR::2014-10-21 20:24:05,303::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238228::ERROR::2014-10-21 20:24:05,304::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238243::ERROR::2014-10-21 20:24:05,600::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238244::ERROR::2014-10-21 20:24:05,633::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238245::ERROR::2014-10-21 20:24:05,656::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238256::ERROR::2014-10-21 20:24:15,696::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238257::ERROR::2014-10-21 20:24:15,770::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238258::ERROR::2014-10-21 20:24:15,793::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-19::ERROR::2014-10-21 20:24:19,848::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain bcfa7ec4-5278-44d8-9f31-682f2d9de91d monitoring information
/var/log/vdsm/vdsm.log:Thread-238269::ERROR::2014-10-21 20:24:25,789::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238270::ERROR::2014-10-21 20:24:25,863::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238271::ERROR::2014-10-21 20:24:25,885::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-30::ERROR::2014-10-21 20:24:28,402::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 68aad705-7c9b-427a-a84c-6f32f23675b3
/var/log/vdsm/vdsm.log:Thread-30::ERROR::2014-10-21 20:24:28,402::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 68aad705-7c9b-427a-a84c-6f32f23675b3
/var/log/vdsm/vdsm.log:Thread-238285::ERROR::2014-10-21 20:24:35,871::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238286::ERROR::2014-10-21 20:24:35,947::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238287::ERROR::2014-10-21 20:24:35,968::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238296::ERROR::2014-10-21 20:24:45,957::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238297::ERROR::2014-10-21 20:24:46,029::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238298::ERROR::2014-10-21 20:24:46,050::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238307::ERROR::2014-10-21 20:24:56,043::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238308::ERROR::2014-10-21 20:24:56,117::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238309::ERROR::2014-10-21 20:24:56,139::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238318::ERROR::2014-10-21 20:25:06,138::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238319::ERROR::2014-10-21 20:25:06,209::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238320::ERROR::2014-10-21 20:25:06,231::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238331::ERROR::2014-10-21 20:25:16,291::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238332::ERROR::2014-10-21 20:25:16,345::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238333::ERROR::2014-10-21 20:25:16,382::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238343::ERROR::2014-10-21 20:25:25,534::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238344::ERROR::2014-10-21 20:25:25,560::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238345::ERROR::2014-10-21 20:25:25,612::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238354::ERROR::2014-10-21 20:25:35,722::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238355::ERROR::2014-10-21 20:25:35,735::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238356::ERROR::2014-10-21 20:25:35,788::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238374::ERROR::2014-10-21 20:25:46,072::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238375::ERROR::2014-10-21 20:25:46,092::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238376::ERROR::2014-10-21 20:25:46,115::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238389::ERROR::2014-10-21 20:25:56,154::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238390::ERROR::2014-10-21 20:25:56,180::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238391::ERROR::2014-10-21 20:25:56,202::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238400::ERROR::2014-10-21 20:26:06,249::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238401::ERROR::2014-10-21 20:26:06,272::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238402::ERROR::2014-10-21 20:26:06,295::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238411::ERROR::2014-10-21 20:26:16,344::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238412::ERROR::2014-10-21 20:26:16,368::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238413::ERROR::2014-10-21 20:26:16,498::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238419::ERROR::2014-10-21 20:26:25,728::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238421::ERROR::2014-10-21 20:26:26,482::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238422::ERROR::2014-10-21 20:26:26,500::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238438::WARNING::2014-10-21 20:26:34,040::fileUtils::149::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/bcfa7ec4-5278-44d8-9f31-682f2d9de91d already exists
/var/log/vdsm/vdsm.log:Thread-238438::WARNING::2014-10-21 20:26:35,572::vm::3971::vm.Vm::(_readPauseCode) vmId=`e68a11c8-1251-4c13-9e3b-3847bbb4fa3d`::_readPauseCode unsupported by libvirt vm
/var/log/vdsm/vdsm.log:Thread-238443::ERROR::2014-10-21 20:26:35,756::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238444::ERROR::2014-10-21 20:26:35,775::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238446::ERROR::2014-10-21 20:26:35,863::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238458::ERROR::2014-10-21 20:26:45,853::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238459::ERROR::2014-10-21 20:26:45,868::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238460::ERROR::2014-10-21 20:26:45,974::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238475::ERROR::2014-10-21 20:26:55,970::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238474::ERROR::2014-10-21 20:26:55,971::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238476::ERROR::2014-10-21 20:26:56,084::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238490::ERROR::2014-10-21 20:27:06,153::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238491::ERROR::2014-10-21 20:27:06,190::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238492::ERROR::2014-10-21 20:27:06,221::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238509::ERROR::2014-10-21 20:27:16,424::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238510::ERROR::2014-10-21 20:27:16,435::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238511::ERROR::2014-10-21 20:27:16,479::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238525::ERROR::2014-10-21 20:27:25,773::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238526::ERROR::2014-10-21 20:27:25,807::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238527::ERROR::2014-10-21 20:27:25,828::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238528::ERROR::2014-10-21 20:27:25,833::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238539::ERROR::2014-10-21 20:27:35,918::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238540::ERROR::2014-10-21 20:27:35,962::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238541::ERROR::2014-10-21 20:27:35,985::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:27:44,206::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238553::ERROR::2014-10-21 20:27:46,087::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238554::ERROR::2014-10-21 20:27:46,146::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238555::ERROR::2014-10-21 20:27:46,165::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238568::ERROR::2014-10-21 20:27:56,338::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238569::ERROR::2014-10-21 20:27:56,365::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238570::ERROR::2014-10-21 20:27:56,392::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:28:02,199::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238585::ERROR::2014-10-21 20:28:06,461::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238586::ERROR::2014-10-21 20:28:06,487::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238587::ERROR::2014-10-21 20:28:06,498::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238600::ERROR::2014-10-21 20:28:15,727::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238601::ERROR::2014-10-21 20:28:15,750::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238602::ERROR::2014-10-21 20:28:15,757::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:28:17,669::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238611::ERROR::2014-10-21 20:28:25,825::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238612::ERROR::2014-10-21 20:28:26,004::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238613::ERROR::2014-10-21 20:28:26,007::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:28:33,341::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238629::ERROR::2014-10-21 20:28:35,986::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238630::ERROR::2014-10-21 20:28:36,115::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238631::ERROR::2014-10-21 20:28:36,138::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238645::ERROR::2014-10-21 20:28:46,082::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238646::ERROR::2014-10-21 20:28:46,203::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238647::ERROR::2014-10-21 20:28:46,224::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:28:48,584::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238662::ERROR::2014-10-21 20:28:56,181::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238663::ERROR::2014-10-21 20:28:56,287::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238664::ERROR::2014-10-21 20:28:56,309::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:29:04,006::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238678::ERROR::2014-10-21 20:29:06,350::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238679::ERROR::2014-10-21 20:29:06,423::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238680::ERROR::2014-10-21 20:29:06,463::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238686::ERROR::2014-10-21 20:29:15,567::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238688::ERROR::2014-10-21 20:29:15,655::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238689::ERROR::2014-10-21 20:29:15,695::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:29:19,291::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238702::ERROR::2014-10-21 20:29:25,673::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238703::ERROR::2014-10-21 20:29:25,745::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238704::ERROR::2014-10-21 20:29:25,781::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-30::ERROR::2014-10-21 20:29:34,310::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 68aad705-7c9b-427a-a84c-6f32f23675b3
/var/log/vdsm/vdsm.log:Thread-30::ERROR::2014-10-21 20:29:34,314::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 68aad705-7c9b-427a-a84c-6f32f23675b3
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:29:34,734::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238719::ERROR::2014-10-21 20:29:35,782::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238720::ERROR::2014-10-21 20:29:35,838::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238721::ERROR::2014-10-21 20:29:35,862::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238735::ERROR::2014-10-21 20:29:45,938::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238736::ERROR::2014-10-21 20:29:46,008::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238737::ERROR::2014-10-21 20:29:46,026::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238534::ERROR::2014-10-21 20:29:50,058::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238750::ERROR::2014-10-21 20:29:56,174::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238751::ERROR::2014-10-21 20:29:56,223::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
/var/log/vdsm/vdsm.log:Thread-238752::ERROR::2014-10-21 20:29:56,248::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
