1:01 p.m.
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.2/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.2/510/
Build Number: 510
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #510
[Roy Golan] Introduce openshift-on-ovirt
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 006_network_by_label.assign_hosts_network_label
Error Message:
status: 409
reason: Conflict
detail: Cannot add Label. Another Setup Networks or Host Refresh process in progress on
the host. Please try later.
-------------------- >> begin captured logging << --------------------
lago.utils: ERROR: Error while running thread
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
_ret_via_queue
queue.put({'return': func()})
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/006_network_by_label.py",
line 57, in _assign_host_network_label
host_nic=nic
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py",
line 16231, in add
headers={"Correlation-Id":correlation_id, "Expect":expect}
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line
79, in add
return self.request('POST', url, body, headers, cls=cls)
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line
122, in request
persistent_auth=self.__persistent_auth
File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
line 79, in do_request
persistent_auth)
File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
line 162, in __do_request
raise errors.RequestError(response_code, response_reason, response_body)
RequestError:
status: 409
reason: Conflict
detail: Cannot add Label. Another Setup Networks or Host Refresh process in progress on
the host. Please try later.
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in
wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 69, in
wrapper
return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs)
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/006_network_by_label.py",
line 65, in assign_hosts_network_label
nt.assert_true(all(vt.join_all()))
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
_ret_via_queue
queue.put({'return': func()})
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/006_network_by_label.py",
line 57, in _assign_host_network_label
host_nic=nic
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py",
line 16231, in add
headers={"Correlation-Id":correlation_id, "Expect":expect}
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line
79, in add
return self.request('POST', url, body, headers, cls=cls)
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line
122, in request
persistent_auth=self.__persistent_auth
File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
line 79, in do_request
persistent_auth)
File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
line 162, in __do_request
raise errors.RequestError(response_code, response_reason, response_body)
'\r\nstatus: 409\r\nreason: Conflict\r\ndetail: Cannot add Label. Another Setup
Networks or Host Refresh process in progress on the host. Please try
later.\n-------------------- >> begin captured logging <<
--------------------\nlago.utils: ERROR: Error while running thread\nTraceback (most
recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py",
line 58, in _ret_via_queue\n queue.put({\'return\': func()})\n File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/006_network_by_label.py",
line 57, in _assign_host_network_label\n host_nic=nic\n File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line
16231, in add\n headers={"Correlation-Id":correlation_id,
"Expect":expect}\n File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 79, in
add\n return self.request(\'POST\', url, body, headers, cls=cls)\n File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122,
in request\n persistent_auth=self.__persistent_auth\n File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
line 79, in do_request\n persistent_auth)\n File
"/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
line 162, in __do_request\n raise errors.RequestError(response_code, response_reason,
response_body)\nRequestError: \r\nstatus: 409\r\nreason: Conflict\r\ndetail: Cannot add
Label. Another Setup Networks or Host Refresh process in progress on the host. Please try
later.\n--------------------- >> end captured logging <<
---------------------'
9:06 p.m.
New subject: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.2 - Build # 511
- Still Failing!
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.2/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.2/511/
Build Number: 511
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #510
[Roy Golan] Introduce openshift-on-ovirt
Changes for Build #511
[Roy Golan] Introduce openshift-on-ovirt
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 008_restart_he_vm.restart_he_vm
Error Message:
could not parse JSON: {"1": {"conf_on_shared_storage": true,
"live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2274 (Sun Aug 26
14:05:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2275 (Sun Aug 26 14:05:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"3a741d0e", "local_conf_timestamp": 2275, "host-ts": 2274},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2266 (Sun Aug 26
14:05:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2267 (Sun Aug 26 14:05:38
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"en
-------------------- >> begin captured logging << --------------------
lago.ssh: DEBUG: start task:8c4ed11c-bc1c-49c8-8110-8626677a50ca:Get ssh client for
lago-he-basic-suite-4-2-host-1:
lago.ssh: DEBUG: end task:8c4ed11c-bc1c-49c8-8110-8626677a50ca:Get ssh client for
lago-he-basic-suite-4-2-host-1:
lago.ssh: DEBUG: Running 4a437212 on lago-he-basic-suite-4-2-host-1: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 4a437212 on lago-he-basic-suite-4-2-host-1 returned with 0
lago.ssh: DEBUG: Command 4a437212 on lago-he-basic-suite-4-2-host-1 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2107 (Sun Aug 26
14:02:58 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2107 (Sun Aug 26 14:02:58
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"health": "good",
"vm": "up", "detail": "Up"}, "score":
3400, "stopped": false, "maintenance": false, "crc32":
"fdf121ef", "local_conf_timestamp": 2107, "host-ts": 2107},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2116 (Sun Aug 26
14:03:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2117 (Sun Aug 26 14:03:08
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "df7cb02f",
"local_conf_timestamp": 2117, "host-ts": 2116},
"global_maintenance": true}
root: INFO: * Shutting down HE VM on host: lago-he-basic-suite-4-2-host-0
lago.ssh: DEBUG: start task:344fccf4-6b8e-406d-a5d7-3dcc2bf664a8:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:344fccf4-6b8e-406d-a5d7-3dcc2bf664a8:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 4abd7a3a on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-shutdown
lago.ssh: DEBUG: Command 4abd7a3a on lago-he-basic-suite-4-2-host-0 returned with 0
root: INFO: * Command succeeded
root: INFO: * Waiting for VM to be down...
lago.ssh: DEBUG: start task:8d831cb7-0035-4e95-ae82-bc1c3e136891:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:8d831cb7-0035-4e95-ae82-bc1c3e136891:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 4b534ace on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 4b534ace on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 4b534ace on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2117 (Sun Aug 26
14:03:09 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2118 (Sun Aug 26 14:03:09
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"health": "good",
"vm": "up", "detail": "Up"}, "score":
3400, "stopped": false, "maintenance": false, "crc32":
"4623b12e", "local_conf_timestamp": 2118, "host-ts": 2117},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2116 (Sun Aug 26
14:03:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2117 (Sun Aug 26 14:03:08
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "df7cb02f",
"local_conf_timestamp": 2117, "host-ts": 2116},
"global_maintenance": true}
lago.ssh: DEBUG: start task:1098018c-f628-4028-b0c3-9a6a3287f1b1:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:1098018c-f628-4028-b0c3-9a6a3287f1b1:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 51e1c168 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 51e1c168 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 51e1c168 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2127 (Sun Aug 26
14:03:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2127 (Sun Aug 26 14:03:19
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm":
"down_unexpected", "detail": "Down"}, "score":
3000, "stopped": false, "maintenance": false, "crc32":
"afa65c75", "local_conf_timestamp": 2127, "host-ts": 2127},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2126 (Sun Aug 26
14:03:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2126 (Sun Aug 26 14:03:17
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "9effa9f6",
"local_conf_timestamp": 2126, "host-ts": 2126},
"global_maintenance": true}
root: INFO: * VM is down.
root: INFO: * Stopping services...
lago.ssh: DEBUG: start task:3e1b8bd7-4f95-448b-84d0-39468d7d35c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:3e1b8bd7-4f95-448b-84d0-39468d7d35c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 525853f0 on lago-he-basic-suite-4-2-host-0: systemctl stop vdsmd
ovirt-ha-broker ovirt-ha-agent
lago.ssh: DEBUG: Command 525853f0 on lago-he-basic-suite-4-2-host-0 returned with 0
root: INFO: * Starting services...
lago.ssh: DEBUG: start task:62250dc4-1c20-4a0e-aa33-a70bda976541:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:62250dc4-1c20-4a0e-aa33-a70bda976541:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 5b86f6f2 on lago-he-basic-suite-4-2-host-0: systemctl start vdsmd
ovirt-ha-broker ovirt-ha-agent
lago.ssh: DEBUG: Command 5b86f6f2 on lago-he-basic-suite-4-2-host-0 returned with 0
root: INFO: * Waiting for agent to be ready...
lago.ssh: DEBUG: start task:02d149e1-b0cf-43c7-aae9-e803d65a7abe:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:02d149e1-b0cf-43c7-aae9-e803d65a7abe:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 5d765b60 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 5d765b60 on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 5d765b60 on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:4a7a6d4f-1d74-488f-bd83-275c4845e2c3:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:4a7a6d4f-1d74-488f-bd83-275c4845e2c3:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 63cf3e46 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 63cf3e46 on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 63cf3e46 on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:2e3309d0-b701-4aab-b0ff-e63603602ec8:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:2e3309d0-b701-4aab-b0ff-e63603602ec8:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 65f19bba on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 65f19bba on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 65f19bba on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:b1c232ea-242a-46db-8678-bdec75eda021:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:b1c232ea-242a-46db-8678-bdec75eda021:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 681c6e60 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 681c6e60 on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 681c6e60 on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:00a3211c-bc3f-4fe6-bc9b-4674e5def441:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:00a3211c-bc3f-4fe6-bc9b-4674e5def441:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 6a448a06 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 6a448a06 on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 6a448a06 on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:f074443d-d9c8-41cf-a414-f5afa3dd390c:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:f074443d-d9c8-41cf-a414-f5afa3dd390c:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 6c63a7a4 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 6c63a7a4 on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 6c63a7a4 on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:44e70a33-7ac8-407e-91d3-703380ea8114:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:44e70a33-7ac8-407e-91d3-703380ea8114:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 6e8069e6 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 6e8069e6 on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 6e8069e6 on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:ac830e90-8461-4575-bd1e-b35832b7c7bf:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:ac830e90-8461-4575-bd1e-b35832b7c7bf:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 70a0250e on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 70a0250e on lago-he-basic-suite-4-2-host-0 returned with 1
lago.ssh: DEBUG: Command 70a0250e on lago-he-basic-suite-4-2-host-0 output:
The hosted engine configuration has not been retrieved from shared storage. Please ensure
that ovirt-ha-agent is running and the storage server is reachable.
lago.ssh: DEBUG: start task:baa6f0d0-3ea7-4ef1-826e-2b995c888019:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:baa6f0d0-3ea7-4ef1-826e-2b995c888019:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 72c118ac on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status
lago.ssh: DEBUG: Command 72c118ac on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 72c118ac on lago-he-basic-suite-4-2-host-0 output:
!! Cluster is in GLOBAL MAINTENANCE mode !!
--== Host 1 status ==--
conf_on_shared_storage : True
Status up-to-date : True
Hostname : lago-he-basic-suite-4-2-host-0.lago.local
Host ID : 1
Engine status : {"reason": "bad vm status",
"health": "bad", "vm": "down", "detail":
"Down"}
Score : 0
stopped : False
Local maintenance : False
crc32 : 4c6fdf7c
local_conf_timestamp : 2185
Host timestamp : 2184
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=2184 (Sun Aug 26 14:04:16 2018)
host-id=1
score=0
vm_conf_refresh_time=2185 (Sun Aug 26 14:04:16 2018)
conf_on_shared_storage=True
maintenance=False
state=ReinitializeFSM
stopped=False
--== Host 2 status ==--
conf_on_shared_storage : True
Status up-to-date : False
Hostname : lago-he-basic-suite-4-2-host-1
Host ID : 2
Engine status : unknown stale-data
Score : 3400
stopped : False
Local maintenance : False
crc32 : c573d227
local_conf_timestamp : 2176
Host timestamp : 2176
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=2176 (Sun Aug 26 14:04:07 2018)
host-id=2
score=3400
vm_conf_refresh_time=2176 (Sun Aug 26 14:04:07 2018)
conf_on_shared_storage=True
maintenance=False
state=GlobalMaintenance
stopped=False
!! Cluster is in GLOBAL MAINTENANCE mode !!
root: INFO: * Agent is ready.
root: INFO: * Starting VM...
lago.ssh: DEBUG: start task:8bac5eac-9415-4098-9f66-f6ba3bfe02c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:8bac5eac-9415-4098-9f66-f6ba3bfe02c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 7355ed88 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-start
lago.ssh: DEBUG: Command 7355ed88 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 7355ed88 on lago-he-basic-suite-4-2-host-0 output:
VM exists and is down, cleaning up and restarting
root: INFO: * Command succeeded
root: INFO: * Waiting for VM to be UP...
lago.ssh: DEBUG: start task:5fbcc916-a582-4a7b-95ba-3347bf9f46ac:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:5fbcc916-a582-4a7b-95ba-3347bf9f46ac:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 74ab61ea on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 74ab61ea on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 74ab61ea on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2184 (Sun Aug 26
14:04:16 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=2185 (Sun Aug 26 14:04:16
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "down",
"detail": "Down"}, "score": 0, "stopped": false,
"maintenance": false, "crc32": "4c6fdf7c",
"local_conf_timestamp": 2185, "host-ts": 2184}, "2":
{"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2176 (Sun Aug 26
14:04:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2176 (Sun Aug 26 14:04:07
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c573d227",
"local_conf_timestamp": 2176, "host-ts": 2176},
"global_maintenance": true}
lago.ssh: DEBUG: start task:ca8c1e4f-693f-4dd3-a170-817d3eb762f4:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:ca8c1e4f-693f-4dd3-a170-817d3eb762f4:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 7b21f624 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 7b21f624 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 7b21f624 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2194 (Sun Aug 26
14:04:25 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2194 (Sun Aug 26 14:04:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"d681be2e", "local_conf_timestamp": 2194, "host-ts": 2194},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2186 (Sun Aug 26
14:04:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2186 (Sun Aug 26 14:04:17
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "29e75e54",
"local_conf_timestamp": 2186, "host-ts": 2186},
"global_maintenance": true}
root: INFO: * VM is UP.
root: INFO: * Waiting for engine to start...
lago.ssh: DEBUG: start task:745ecf00-06f9-4897-bd8a-0ba33de9867c:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:745ecf00-06f9-4897-bd8a-0ba33de9867c:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 7b9a9e30 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 7b9a9e30 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 7b9a9e30 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2194 (Sun Aug 26
14:04:25 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2194 (Sun Aug 26 14:04:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"d681be2e", "local_conf_timestamp": 2194, "host-ts": 2194},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2186 (Sun Aug 26
14:04:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2186 (Sun Aug 26 14:04:17
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "29e75e54",
"local_conf_timestamp": 2186, "host-ts": 2186},
"global_maintenance": true}
lago.ssh: DEBUG: start task:bc151d0d-e429-45ec-b326-c4e57b04f706:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:bc151d0d-e429-45ec-b326-c4e57b04f706:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 8206d8e2 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 8206d8e2 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 8206d8e2 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2204 (Sun Aug 26
14:04:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2205 (Sun Aug 26 14:04:36
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"34bf6954", "local_conf_timestamp": 2205, "host-ts": 2204},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2196 (Sun Aug 26
14:04:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2197 (Sun Aug 26 14:04:28
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "4c9c7607",
"local_conf_timestamp": 2197, "host-ts": 2196},
"global_maintenance": true}
lago.ssh: DEBUG: start task:8f21eb76-51e0-4706-a017-4b06b617459f:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:8f21eb76-51e0-4706-a017-4b06b617459f:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 845130a2 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 845130a2 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 845130a2 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2214 (Sun Aug 26
14:04:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2215 (Sun Aug 26 14:04:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"9152fdda", "local_conf_timestamp": 2215, "host-ts": 2214},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2206 (Sun Aug 26
14:04:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2206 (Sun Aug 26 14:04:37
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "5c524054",
"local_conf_timestamp": 2206, "host-ts": 2206},
"global_maintenance": true}
lago.ssh: DEBUG: start task:126ecf00-037c-40ed-84a6-3b75751980dc:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:126ecf00-037c-40ed-84a6-3b75751980dc:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 86951798 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 86951798 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 86951798 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2214 (Sun Aug 26
14:04:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2215 (Sun Aug 26 14:04:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"9152fdda", "local_conf_timestamp": 2215, "host-ts": 2214},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2206 (Sun Aug 26
14:04:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2206 (Sun Aug 26 14:04:37
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "5c524054",
"local_conf_timestamp": 2206, "host-ts": 2206},
"global_maintenance": true}
lago.ssh: DEBUG: start task:e1f2dd2f-6ec9-4f89-af40-88105a9034e7:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:e1f2dd2f-6ec9-4f89-af40-88105a9034e7:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 891db2a4 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 891db2a4 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 891db2a4 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2214 (Sun Aug 26
14:04:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2215 (Sun Aug 26 14:04:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"9152fdda", "local_conf_timestamp": 2215, "host-ts": 2214},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2206 (Sun Aug 26
14:04:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2206 (Sun Aug 26 14:04:37
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "5c524054",
"local_conf_timestamp": 2206, "host-ts": 2206},
"global_maintenance": true}
lago.ssh: DEBUG: start task:4c0118ba-3f9f-4e7a-a34e-c86745110252:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:4c0118ba-3f9f-4e7a-a34e-c86745110252:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 8b66f4bc on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 8b66f4bc on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 8b66f4bc on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2224 (Sun Aug 26
14:04:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2224 (Sun Aug 26 14:04:56
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"06c7c260", "local_conf_timestamp": 2224, "host-ts": 2224},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2216 (Sun Aug 26
14:04:47 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2217 (Sun Aug 26 14:04:48
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "39296807",
"local_conf_timestamp": 2217, "host-ts": 2216},
"global_maintenance": true}
lago.ssh: DEBUG: start task:29c432bf-3e91-4b02-979e-bc4ada4b5e31:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:29c432bf-3e91-4b02-979e-bc4ada4b5e31:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 8db71878 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 8db71878 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 8db71878 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2224 (Sun Aug 26
14:04:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2224 (Sun Aug 26 14:04:56
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"06c7c260", "local_conf_timestamp": 2224, "host-ts": 2224},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2216 (Sun Aug 26
14:04:47 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2217 (Sun Aug 26 14:04:48
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "39296807",
"local_conf_timestamp": 2217, "host-ts": 2216},
"global_maintenance": true}
lago.ssh: DEBUG: start task:8ca20943-46ec-468f-866b-926e9c3e7ea6:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:8ca20943-46ec-468f-866b-926e9c3e7ea6:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 90019b26 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 90019b26 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 90019b26 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2234 (Sun Aug 26
14:05:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2234 (Sun Aug 26 14:05:05
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"14bbdaa5", "local_conf_timestamp": 2234, "host-ts": 2234},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2226 (Sun Aug 26
14:04:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2226 (Sun Aug 26 14:04:57
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78aa71de",
"local_conf_timestamp": 2226, "host-ts": 2226},
"global_maintenance": true}
lago.ssh: DEBUG: start task:8fca79b1-660e-49e2-83b8-3fc306aa140c:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:8fca79b1-660e-49e2-83b8-3fc306aa140c:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 9269ceba on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 9269ceba on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 9269ceba on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2234 (Sun Aug 26
14:05:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2234 (Sun Aug 26 14:05:05
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"14bbdaa5", "local_conf_timestamp": 2234, "host-ts": 2234},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2226 (Sun Aug 26
14:04:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2226 (Sun Aug 26 14:04:57
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78aa71de",
"local_conf_timestamp": 2226, "host-ts": 2226},
"global_maintenance": true}
lago.ssh: DEBUG: start task:758d9896-bd87-4dbf-8748-35e94f75dd00:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:758d9896-bd87-4dbf-8748-35e94f75dd00:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 94aa1d2e on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 94aa1d2e on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 94aa1d2e on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2234 (Sun Aug 26
14:05:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2234 (Sun Aug 26 14:05:05
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"14bbdaa5", "local_conf_timestamp": 2234, "host-ts": 2234},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2226 (Sun Aug 26
14:04:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2226 (Sun Aug 26 14:04:57
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78aa71de",
"local_conf_timestamp": 2226, "host-ts": 2226},
"global_maintenance": true}
lago.ssh: DEBUG: start task:360edee2-2fd9-4382-8331-03ae68226f1b:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:360edee2-2fd9-4382-8331-03ae68226f1b:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 971268be on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 971268be on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 971268be on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2244 (Sun Aug 26
14:05:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2245 (Sun Aug 26 14:05:16
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"1cc8a068", "local_conf_timestamp": 2245, "host-ts": 2244},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2236 (Sun Aug 26
14:05:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2236 (Sun Aug 26 14:05:07
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "6ad6691b",
"local_conf_timestamp": 2236, "host-ts": 2236},
"global_maintenance": true}
lago.ssh: DEBUG: start task:7456e721-7a0a-46da-bbb0-770bbc09ac54:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:7456e721-7a0a-46da-bbb0-770bbc09ac54:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 997f8ad2 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 997f8ad2 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 997f8ad2 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2244 (Sun Aug 26
14:05:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2245 (Sun Aug 26 14:05:16
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"1cc8a068", "local_conf_timestamp": 2245, "host-ts": 2244},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2236 (Sun Aug 26
14:05:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2236 (Sun Aug 26 14:05:07
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "6ad6691b",
"local_conf_timestamp": 2236, "host-ts": 2236},
"global_maintenance": true}
lago.ssh: DEBUG: start task:536fbae0-0bd0-478e-a20e-325a685274fa:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:536fbae0-0bd0-478e-a20e-325a685274fa:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 9be04cf8 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 9be04cf8 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 9be04cf8 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2244 (Sun Aug 26
14:05:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2245 (Sun Aug 26 14:05:16
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"1cc8a068", "local_conf_timestamp": 2245, "host-ts": 2244},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2236 (Sun Aug 26
14:05:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2236 (Sun Aug 26 14:05:07
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "6ad6691b",
"local_conf_timestamp": 2236, "host-ts": 2236},
"global_maintenance": true}
lago.ssh: DEBUG: start task:e916b2dc-2d57-4571-83ea-a03b5c863fb9:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:e916b2dc-2d57-4571-83ea-a03b5c863fb9:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running 9e63cc5c on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command 9e63cc5c on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 9e63cc5c on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2254 (Sun Aug 26
14:05:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2255 (Sun Aug 26 14:05:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"1e8c2c84", "local_conf_timestamp": 2255, "host-ts": 2254},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2246 (Sun Aug 26
14:05:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2247 (Sun Aug 26 14:05:18
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "62a513d6",
"local_conf_timestamp": 2247, "host-ts": 2246},
"global_maintenance": true}
lago.ssh: DEBUG: start task:361ba153-6e78-49de-990c-9f8461794e77:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:361ba153-6e78-49de-990c-9f8461794e77:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running a0e98318 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command a0e98318 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command a0e98318 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2254 (Sun Aug 26
14:05:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2255 (Sun Aug 26 14:05:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"1e8c2c84", "local_conf_timestamp": 2255, "host-ts": 2254},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2246 (Sun Aug 26
14:05:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2247 (Sun Aug 26 14:05:18
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "62a513d6",
"local_conf_timestamp": 2247, "host-ts": 2246},
"global_maintenance": true}
lago.ssh: DEBUG: start task:ad5a7a91-e1ae-47a9-bee1-80971fcc5f22:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:ad5a7a91-e1ae-47a9-bee1-80971fcc5f22:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running a354845e on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command a354845e on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command a354845e on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2264 (Sun Aug 26
14:05:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2265 (Sun Aug 26 14:05:36
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"280805cb", "local_conf_timestamp": 2265, "host-ts": 2264},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2256 (Sun Aug 26
14:05:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2256 (Sun Aug 26 14:05:27
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "07de3b85",
"local_conf_timestamp": 2256, "host-ts": 2256},
"global_maintenance": true}
lago.ssh: DEBUG: start task:b5e00fb8-336a-4bc8-82d1-7ab62556304b:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:b5e00fb8-336a-4bc8-82d1-7ab62556304b:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running a5a62c44 on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command a5a62c44 on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command a5a62c44 on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2264 (Sun Aug 26
14:05:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2265 (Sun Aug 26 14:05:36
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"280805cb", "local_conf_timestamp": 2265, "host-ts": 2264},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2256 (Sun Aug 26
14:05:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2256 (Sun Aug 26 14:05:27
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "07de3b85",
"local_conf_timestamp": 2256, "host-ts": 2256},
"global_maintenance": true}
lago.ssh: DEBUG: start task:c942e768-03b4-4e0a-b5a1-8f9bee1f1541:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:c942e768-03b4-4e0a-b5a1-8f9bee1f1541:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running a7ff984a on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command a7ff984a on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command a7ff984a on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2264 (Sun Aug 26
14:05:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2265 (Sun Aug 26 14:05:36
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"280805cb", "local_conf_timestamp": 2265, "host-ts": 2264},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2256 (Sun Aug 26
14:05:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2256 (Sun Aug 26 14:05:27
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "07de3b85",
"local_conf_timestamp": 2256, "host-ts": 2256},
"global_maintenance": true}
lago.ssh: DEBUG: start task:318bc110-f27d-496e-b518-863d03725adf:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: end task:318bc110-f27d-496e-b518-863d03725adf:Get ssh client for
lago-he-basic-suite-4-2-host-0:
lago.ssh: DEBUG: Running aa43905c on lago-he-basic-suite-4-2-host-0: hosted-engine
--vm-status --json
lago.ssh: DEBUG: Command aa43905c on lago-he-basic-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command aa43905c on lago-he-basic-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2274 (Sun Aug 26
14:05:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2275 (Sun Aug 26 14:05:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"3a741d0e", "local_conf_timestamp": 2275, "host-ts": 2274},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2266 (Sun Aug 26
14:05:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2267 (Sun Aug 26 14:05:38
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"en
ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at
0x7f3c24244cf8>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in
assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 187, in <lambda>
for k, v in _get_he_status(host).items()
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 128, in _get_he_status
raise RuntimeError('could not parse JSON: %s' % ret.out)
RuntimeError: could not parse JSON: {"1": {"conf_on_shared_storage":
true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2274 (Sun Aug 26
14:05:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2275 (Sun Aug 26 14:05:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"3a741d0e", "local_conf_timestamp": 2275, "host-ts": 2274},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2266 (Sun Aug 26
14:05:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2267 (Sun Aug 26 14:05:38
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"en
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in
wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 53, in restart_he_vm
_wait_for_engine_health(host)
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 185, in _wait_for_engine_health
testlib.assert_true_within_long(lambda: any(
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in
assert_true_within_long
assert_equals_within_long(func, True, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in
assert_equals_within_long
func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in
assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 187, in <lambda>
for k, v in _get_he_status(host).items()
File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 128, in _get_he_status
raise RuntimeError('could not parse JSON: %s' % ret.out)
'could not parse JSON: {"1": {"conf_on_shared_storage": true,
"live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2274 (Sun Aug 26
14:05:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2275 (Sun Aug 26 14:05:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"3a741d0e", "local_conf_timestamp": 2275, "host-ts": 2274},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2266 (Sun Aug 26
14:05:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2267 (Sun Aug 26 14:05:38
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"en\n-------------------- >> begin captured logging <<
--------------------\nlago.ssh: DEBUG: start task:8c4ed11c-bc1c-49c8-8110-8626677a50ca:Get
ssh client for lago-he-basic-suite-4-2-host-1:\nlago.ssh: DEBUG: end
task:8c4ed11c-bc1c-49c8-8110-8626677a50ca:Get ssh client for
lago-he-basic-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 4a437212 on
lago-he-basic-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
4a437212 on lago-he-basic-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command
4a437212 on lago-he-basic-suite-4-2-host-1 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2107 (Sun Aug 26
14:02:58 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2107 (Sun Aug 26 14:02:58
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"health": "good",
"vm": "up", "detail": "Up"}, "score":
3400, "stopped": false, "maintenance": false, "crc32":
"fdf121ef", "local_conf_timestamp": 2107, "host-ts": 2107},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2116 (Sun Aug 26
14:03:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2117 (Sun Aug 26 14:03:08
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "df7cb02f",
"local_conf_timestamp": 2117, "host-ts": 2116},
"global_maintenance": true}\n\nroot: INFO: * Shutting down HE VM on host:
lago-he-basic-suite-4-2-host-0\nlago.ssh: DEBUG: start
task:344fccf4-6b8e-406d-a5d7-3dcc2bf664a8:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:344fccf4-6b8e-406d-a5d7-3dcc2bf664a8:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4abd7a3a on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-shutdown\nlago.ssh: DEBUG: Command
4abd7a3a on lago-he-basic-suite-4-2-host-0 returned with 0\nroot: INFO: * Command
succeeded\nroot: INFO: * Waiting for VM to be down...\nlago.ssh: DEBUG: start
task:8d831cb7-0035-4e95-ae82-bc1c3e136891:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:8d831cb7-0035-4e95-ae82-bc1c3e136891:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4b534ace on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
4b534ace on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
4b534ace on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2117 (Sun Aug 26
14:03:09 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2118 (Sun Aug 26 14:03:09
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"health": "good",
"vm": "up", "detail": "Up"}, "score":
3400, "stopped": false, "maintenance": false, "crc32":
"4623b12e", "local_conf_timestamp": 2118, "host-ts": 2117},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2116 (Sun Aug 26
14:03:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2117 (Sun Aug 26 14:03:08
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "df7cb02f",
"local_conf_timestamp": 2117, "host-ts": 2116},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:1098018c-f628-4028-b0c3-9a6a3287f1b1:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:1098018c-f628-4028-b0c3-9a6a3287f1b1:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 51e1c168 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
51e1c168 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
51e1c168 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2127 (Sun Aug 26
14:03:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2127 (Sun Aug 26 14:03:19
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm":
"down_unexpected", "detail": "Down"}, "score":
3000, "stopped": false, "maintenance": false, "crc32":
"afa65c75", "local_conf_timestamp": 2127, "host-ts": 2127},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2126 (Sun Aug 26
14:03:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2126 (Sun Aug 26 14:03:17
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "9effa9f6",
"local_conf_timestamp": 2126, "host-ts": 2126},
"global_maintenance": true}\n\nroot: INFO: * VM is down.\nroot: INFO: *
Stopping services...\nlago.ssh: DEBUG: start task:3e1b8bd7-4f95-448b-84d0-39468d7d35c4:Get
ssh client for lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:3e1b8bd7-4f95-448b-84d0-39468d7d35c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 525853f0 on
lago-he-basic-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker
ovirt-ha-agent\nlago.ssh: DEBUG: Command 525853f0 on lago-he-basic-suite-4-2-host-0
returned with 0\nroot: INFO: * Starting services...\nlago.ssh: DEBUG: start
task:62250dc4-1c20-4a0e-aa33-a70bda976541:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:62250dc4-1c20-4a0e-aa33-a70bda976541:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5b86f6f2 on
lago-he-basic-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker
ovirt-ha-agent\nlago.ssh: DEBUG: Command 5b86f6f2 on lago-he-basic-suite-4-2-host-0
returned with 0\nroot: INFO: * Waiting for agent to be ready...\nlago.ssh: DEBUG:
start task:02d149e1-b0cf-43c7-aae9-e803d65a7abe:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:02d149e1-b0cf-43c7-aae9-e803d65a7abe:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5d765b60 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
5d765b60 on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
5d765b60 on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:4a7a6d4f-1d74-488f-bd83-275c4845e2c3:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:4a7a6d4f-1d74-488f-bd83-275c4845e2c3:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 63cf3e46 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
63cf3e46 on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
63cf3e46 on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:2e3309d0-b701-4aab-b0ff-e63603602ec8:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:2e3309d0-b701-4aab-b0ff-e63603602ec8:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 65f19bba on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
65f19bba on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
65f19bba on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:b1c232ea-242a-46db-8678-bdec75eda021:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:b1c232ea-242a-46db-8678-bdec75eda021:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 681c6e60 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
681c6e60 on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
681c6e60 on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:00a3211c-bc3f-4fe6-bc9b-4674e5def441:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:00a3211c-bc3f-4fe6-bc9b-4674e5def441:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6a448a06 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
6a448a06 on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
6a448a06 on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:f074443d-d9c8-41cf-a414-f5afa3dd390c:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:f074443d-d9c8-41cf-a414-f5afa3dd390c:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6c63a7a4 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
6c63a7a4 on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
6c63a7a4 on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:44e70a33-7ac8-407e-91d3-703380ea8114:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:44e70a33-7ac8-407e-91d3-703380ea8114:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6e8069e6 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
6e8069e6 on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
6e8069e6 on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:ac830e90-8461-4575-bd1e-b35832b7c7bf:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:ac830e90-8461-4575-bd1e-b35832b7c7bf:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 70a0250e on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
70a0250e on lago-he-basic-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command
70a0250e on lago-he-basic-suite-4-2-host-0 output:\n The hosted engine configuration has
not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and
the storage server is reachable.\n\nlago.ssh: DEBUG: start
task:baa6f0d0-3ea7-4ef1-826e-2b995c888019:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:baa6f0d0-3ea7-4ef1-826e-2b995c888019:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 72c118ac on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command
72c118ac on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
72c118ac on lago-he-basic-suite-4-2-host-0 output:\n \n\n!! Cluster is in GLOBAL
MAINTENANCE mode !!\n\n\n\n--== Host 1 status ==--\n\nconf_on_shared_storage :
True\nStatus up-to-date : True\nHostname :
lago-he-basic-suite-4-2-host-0.lago.local\nHost ID : 1\nEngine
status : {"reason": "bad vm status",
"health": "bad", "vm": "down", "detail":
"Down"}\nScore : 0\nstopped
: False\nLocal maintenance : False\ncrc32
: 4c6fdf7c\nlocal_conf_timestamp : 2185\nHost timestamp
: 2184\nExtra metadata (valid at
timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=2184
(Sun Aug 26 14:04:16 2018)\n\thost-id=1\n\tscore=0\n\tvm_conf_refresh_time=2185 (Sun Aug
26 14:04:16
2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=ReinitializeFSM\n\tstopped=False\n\n\n--==
Host 2 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date
: False\nHostname :
lago-he-basic-suite-4-2-host-1\nHost ID : 2\nEngine status
: unknown stale-data\nScore : 3400\nstopped
: False\nLocal maintenance : False\ncrc32
: c573d227\nlocal_conf_timestamp : 2176\nHost
timestamp : 2176\nExtra metadata (valid at
timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=2176
(Sun Aug 26 14:04:07 2018)\n\thost-id=2\n\tscore=3400\n\tvm_conf_refresh_time=2176 (Sun
Aug 26 14:04:07
2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=GlobalMaintenance\n\tstopped=False\n\n\n!!
Cluster is in GLOBAL MAINTENANCE mode !!\n\n\nroot: INFO: * Agent is ready.\nroot:
INFO: * Starting VM...\nlago.ssh: DEBUG: start
task:8bac5eac-9415-4098-9f66-f6ba3bfe02c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:8bac5eac-9415-4098-9f66-f6ba3bfe02c4:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7355ed88 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-start\nlago.ssh: DEBUG: Command
7355ed88 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
7355ed88 on lago-he-basic-suite-4-2-host-0 output:\n VM exists and is down, cleaning up
and restarting\n\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to
be UP...\nlago.ssh: DEBUG: start task:5fbcc916-a582-4a7b-95ba-3347bf9f46ac:Get ssh client
for lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:5fbcc916-a582-4a7b-95ba-3347bf9f46ac:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 74ab61ea on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
74ab61ea on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
74ab61ea on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2184 (Sun Aug 26
14:04:16 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=2185 (Sun Aug 26 14:04:16
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "down",
"detail": "Down"}, "score": 0, "stopped": false,
"maintenance": false, "crc32": "4c6fdf7c",
"local_conf_timestamp": 2185, "host-ts": 2184}, "2":
{"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2176 (Sun Aug 26
14:04:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2176 (Sun Aug 26 14:04:07
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c573d227",
"local_conf_timestamp": 2176, "host-ts": 2176},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:ca8c1e4f-693f-4dd3-a170-817d3eb762f4:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:ca8c1e4f-693f-4dd3-a170-817d3eb762f4:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7b21f624 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
7b21f624 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
7b21f624 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2194 (Sun Aug 26
14:04:25 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2194 (Sun Aug 26 14:04:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"d681be2e", "local_conf_timestamp": 2194, "host-ts": 2194},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2186 (Sun Aug 26
14:04:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2186 (Sun Aug 26 14:04:17
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "29e75e54",
"local_conf_timestamp": 2186, "host-ts": 2186},
"global_maintenance": true}\n\nroot: INFO: * VM is UP.\nroot: INFO: *
Waiting for engine to start...\nlago.ssh: DEBUG: start
task:745ecf00-06f9-4897-bd8a-0ba33de9867c:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:745ecf00-06f9-4897-bd8a-0ba33de9867c:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7b9a9e30 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
7b9a9e30 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
7b9a9e30 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2194 (Sun Aug 26
14:04:25 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2194 (Sun Aug 26 14:04:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"d681be2e", "local_conf_timestamp": 2194, "host-ts": 2194},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2186 (Sun Aug 26
14:04:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2186 (Sun Aug 26 14:04:17
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "29e75e54",
"local_conf_timestamp": 2186, "host-ts": 2186},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:bc151d0d-e429-45ec-b326-c4e57b04f706:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:bc151d0d-e429-45ec-b326-c4e57b04f706:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8206d8e2 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
8206d8e2 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
8206d8e2 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2204 (Sun Aug 26
14:04:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2205 (Sun Aug 26 14:04:36
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"34bf6954", "local_conf_timestamp": 2205, "host-ts": 2204},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2196 (Sun Aug 26
14:04:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2197 (Sun Aug 26 14:04:28
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "4c9c7607",
"local_conf_timestamp": 2197, "host-ts": 2196},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:8f21eb76-51e0-4706-a017-4b06b617459f:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:8f21eb76-51e0-4706-a017-4b06b617459f:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 845130a2 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
845130a2 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
845130a2 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2214 (Sun Aug 26
14:04:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2215 (Sun Aug 26 14:04:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"9152fdda", "local_conf_timestamp": 2215, "host-ts": 2214},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2206 (Sun Aug 26
14:04:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2206 (Sun Aug 26 14:04:37
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "5c524054",
"local_conf_timestamp": 2206, "host-ts": 2206},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:126ecf00-037c-40ed-84a6-3b75751980dc:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:126ecf00-037c-40ed-84a6-3b75751980dc:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 86951798 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
86951798 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
86951798 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2214 (Sun Aug 26
14:04:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2215 (Sun Aug 26 14:04:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"9152fdda", "local_conf_timestamp": 2215, "host-ts": 2214},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2206 (Sun Aug 26
14:04:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2206 (Sun Aug 26 14:04:37
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "5c524054",
"local_conf_timestamp": 2206, "host-ts": 2206},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:e1f2dd2f-6ec9-4f89-af40-88105a9034e7:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:e1f2dd2f-6ec9-4f89-af40-88105a9034e7:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 891db2a4 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
891db2a4 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
891db2a4 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2214 (Sun Aug 26
14:04:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2215 (Sun Aug 26 14:04:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "bad vm
status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"9152fdda", "local_conf_timestamp": 2215, "host-ts": 2214},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2206 (Sun Aug 26
14:04:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2206 (Sun Aug 26 14:04:37
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "5c524054",
"local_conf_timestamp": 2206, "host-ts": 2206},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:4c0118ba-3f9f-4e7a-a34e-c86745110252:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:4c0118ba-3f9f-4e7a-a34e-c86745110252:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8b66f4bc on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
8b66f4bc on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
8b66f4bc on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2224 (Sun Aug 26
14:04:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2224 (Sun Aug 26 14:04:56
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"06c7c260", "local_conf_timestamp": 2224, "host-ts": 2224},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2216 (Sun Aug 26
14:04:47 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2217 (Sun Aug 26 14:04:48
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "39296807",
"local_conf_timestamp": 2217, "host-ts": 2216},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:29c432bf-3e91-4b02-979e-bc4ada4b5e31:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:29c432bf-3e91-4b02-979e-bc4ada4b5e31:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8db71878 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
8db71878 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
8db71878 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2224 (Sun Aug 26
14:04:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2224 (Sun Aug 26 14:04:56
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"06c7c260", "local_conf_timestamp": 2224, "host-ts": 2224},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2216 (Sun Aug 26
14:04:47 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2217 (Sun Aug 26 14:04:48
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "39296807",
"local_conf_timestamp": 2217, "host-ts": 2216},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:8ca20943-46ec-468f-866b-926e9c3e7ea6:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:8ca20943-46ec-468f-866b-926e9c3e7ea6:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 90019b26 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
90019b26 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
90019b26 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2234 (Sun Aug 26
14:05:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2234 (Sun Aug 26 14:05:05
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"14bbdaa5", "local_conf_timestamp": 2234, "host-ts": 2234},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2226 (Sun Aug 26
14:04:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2226 (Sun Aug 26 14:04:57
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78aa71de",
"local_conf_timestamp": 2226, "host-ts": 2226},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:8fca79b1-660e-49e2-83b8-3fc306aa140c:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:8fca79b1-660e-49e2-83b8-3fc306aa140c:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9269ceba on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
9269ceba on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
9269ceba on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2234 (Sun Aug 26
14:05:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2234 (Sun Aug 26 14:05:05
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"14bbdaa5", "local_conf_timestamp": 2234, "host-ts": 2234},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2226 (Sun Aug 26
14:04:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2226 (Sun Aug 26 14:04:57
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78aa71de",
"local_conf_timestamp": 2226, "host-ts": 2226},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:758d9896-bd87-4dbf-8748-35e94f75dd00:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:758d9896-bd87-4dbf-8748-35e94f75dd00:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 94aa1d2e on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
94aa1d2e on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
94aa1d2e on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2234 (Sun Aug 26
14:05:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2234 (Sun Aug 26 14:05:05
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"14bbdaa5", "local_conf_timestamp": 2234, "host-ts": 2234},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2226 (Sun Aug 26
14:04:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2226 (Sun Aug 26 14:04:57
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78aa71de",
"local_conf_timestamp": 2226, "host-ts": 2226},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:360edee2-2fd9-4382-8331-03ae68226f1b:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:360edee2-2fd9-4382-8331-03ae68226f1b:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 971268be on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
971268be on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
971268be on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2244 (Sun Aug 26
14:05:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2245 (Sun Aug 26 14:05:16
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"1cc8a068", "local_conf_timestamp": 2245, "host-ts": 2244},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2236 (Sun Aug 26
14:05:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2236 (Sun Aug 26 14:05:07
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "6ad6691b",
"local_conf_timestamp": 2236, "host-ts": 2236},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:7456e721-7a0a-46da-bbb0-770bbc09ac54:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:7456e721-7a0a-46da-bbb0-770bbc09ac54:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 997f8ad2 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
997f8ad2 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
997f8ad2 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2244 (Sun Aug 26
14:05:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2245 (Sun Aug 26 14:05:16
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"1cc8a068", "local_conf_timestamp": 2245, "host-ts": 2244},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2236 (Sun Aug 26
14:05:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2236 (Sun Aug 26 14:05:07
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "6ad6691b",
"local_conf_timestamp": 2236, "host-ts": 2236},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:536fbae0-0bd0-478e-a20e-325a685274fa:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:536fbae0-0bd0-478e-a20e-325a685274fa:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9be04cf8 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
9be04cf8 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
9be04cf8 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2244 (Sun Aug 26
14:05:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2245 (Sun Aug 26 14:05:16
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32":
"1cc8a068", "local_conf_timestamp": 2245, "host-ts": 2244},
"2": {"conf_on_shared_storage": true, "live-data": false,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2236 (Sun Aug 26
14:05:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2236 (Sun Aug 26 14:05:07
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "6ad6691b",
"local_conf_timestamp": 2236, "host-ts": 2236},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:e916b2dc-2d57-4571-83ea-a03b5c863fb9:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:e916b2dc-2d57-4571-83ea-a03b5c863fb9:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9e63cc5c on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
9e63cc5c on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
9e63cc5c on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2254 (Sun Aug 26
14:05:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2255 (Sun Aug 26 14:05:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"1e8c2c84", "local_conf_timestamp": 2255, "host-ts": 2254},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2246 (Sun Aug 26
14:05:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2247 (Sun Aug 26 14:05:18
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "62a513d6",
"local_conf_timestamp": 2247, "host-ts": 2246},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:361ba153-6e78-49de-990c-9f8461794e77:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:361ba153-6e78-49de-990c-9f8461794e77:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a0e98318 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
a0e98318 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
a0e98318 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2254 (Sun Aug 26
14:05:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2255 (Sun Aug 26 14:05:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"1e8c2c84", "local_conf_timestamp": 2255, "host-ts": 2254},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2246 (Sun Aug 26
14:05:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2247 (Sun Aug 26 14:05:18
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "62a513d6",
"local_conf_timestamp": 2247, "host-ts": 2246},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:ad5a7a91-e1ae-47a9-bee1-80971fcc5f22:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:ad5a7a91-e1ae-47a9-bee1-80971fcc5f22:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a354845e on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
a354845e on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
a354845e on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2264 (Sun Aug 26
14:05:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2265 (Sun Aug 26 14:05:36
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"280805cb", "local_conf_timestamp": 2265, "host-ts": 2264},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2256 (Sun Aug 26
14:05:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2256 (Sun Aug 26 14:05:27
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "07de3b85",
"local_conf_timestamp": 2256, "host-ts": 2256},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:b5e00fb8-336a-4bc8-82d1-7ab62556304b:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:b5e00fb8-336a-4bc8-82d1-7ab62556304b:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a5a62c44 on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
a5a62c44 on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
a5a62c44 on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2264 (Sun Aug 26
14:05:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2265 (Sun Aug 26 14:05:36
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"280805cb", "local_conf_timestamp": 2265, "host-ts": 2264},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2256 (Sun Aug 26
14:05:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2256 (Sun Aug 26 14:05:27
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "07de3b85",
"local_conf_timestamp": 2256, "host-ts": 2256},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:c942e768-03b4-4e0a-b5a1-8f9bee1f1541:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:c942e768-03b4-4e0a-b5a1-8f9bee1f1541:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a7ff984a on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
a7ff984a on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
a7ff984a on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2264 (Sun Aug 26
14:05:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2265 (Sun Aug 26 14:05:36
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"280805cb", "local_conf_timestamp": 2265, "host-ts": 2264},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2256 (Sun Aug 26
14:05:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2256 (Sun Aug 26 14:05:27
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host",
"health": "bad", "vm": "down", "detail":
"unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "07de3b85",
"local_conf_timestamp": 2256, "host-ts": 2256},
"global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:318bc110-f27d-496e-b518-863d03725adf:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: end
task:318bc110-f27d-496e-b518-863d03725adf:Get ssh client for
lago-he-basic-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aa43905c on
lago-he-basic-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command
aa43905c on lago-he-basic-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command
aa43905c on lago-he-basic-suite-4-2-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2274 (Sun Aug 26
14:05:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2275 (Sun Aug 26 14:05:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"3a741d0e", "local_conf_timestamp": 2275, "host-ts": 2274},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2266 (Sun Aug 26
14:05:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2267 (Sun Aug 26 14:05:38
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"en\novirtlago.testlib: ERROR: * Unhandled exception in <function
<lambda> at 0x7f3c24244cf8>\nTraceback (most recent call last):\n File
"/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in
assert_equals_within\n res = func()\n File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 187, in <lambda>\n for k, v in _get_he_status(host).items()\n File
"/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.2/ovirt-system-tests/he-basic-suite-4.2/test-scenarios/008_restart_he_vm.py",
line 128, in _get_he_status\n raise RuntimeError(\'could not parse JSON: %s\' %
ret.out)\nRuntimeError: could not parse JSON: {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2274 (Sun Aug 26
14:05:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2275 (Sun Aug 26 14:05:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-0.lago.local",
"host-id": 1, "engine-status": {"reason": "failed
liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000,
"stopped": false, "maintenance": false, "crc32":
"3a741d0e", "local_conf_timestamp": 2275, "host-ts": 2274},
"2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2266 (Sun Aug 26
14:05:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2267 (Sun Aug 26 14:05:38
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-basic-suite-4-2-host-1", "host-id": 2,
"en\n--------------------- >> end captured logging <<
---------------------'
5:34 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.2 - Build # 512
- Fixed!
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.2/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.2/512/
Build Number: 512
Build Status: Fixed
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #510
[Roy Golan] Introduce openshift-on-ovirt
Changes for Build #511
[Roy Golan] Introduce openshift-on-ovirt
Changes for Build #512
[Roy Golan] Introduce openshift-on-ovirt
-----------------
Failed Tests:
-----------------
All tests passed
2275
days inactive
2276
days old
2 comments
1 participants
participants (1)
-
jenkins＠jenkins.phx.ovirt.org