[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 347 - Failure!
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/347/ Build Number: 347 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:90555ac6-a749-4c6d-97d0-ca20f9ee7e56:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:90555ac6-a749-4c6d-97d0-ca20f9ee7e56:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 4d4a6eae on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4d4a6eae on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 4d4a6eae on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Wed Sep 26 17:40:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4241 (Wed Sep 26 17:40:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "49ba6ea7", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Wed Sep 26 17:40:41 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4247 (Wed Sep 26 17:40:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e8286b5a", "local_conf_timestamp": 4247, "host-ts": 4247}, "global_maintenance": false} lago.ssh: DEBUG: start task:cdd8bc8a-3f38-4e62-9699-7ef8e4864f9a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cdd8bc8a-3f38-4e62-9699-7ef8e4864f9a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4e65c50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 4e65c50e on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:cdb61607-0d9d-4531-931e-e2a2c02f6b15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cdb61607-0d9d-4531-931e-e2a2c02f6b15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4fac57de on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4fac57de on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4fac57de on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Wed Sep 26 17:40:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4251 (Wed Sep 26 17:40:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "95dd2e98", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Wed Sep 26 17:40:41 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4247 (Wed Sep 26 17:40:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e8286b5a", "local_conf_timestamp": 4247, "host-ts": 4247}, "global_maintenance": false} lago.ssh: DEBUG: start task:1fc0139b-3d03-4e0a-8168-69c54559563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1fc0139b-3d03-4e0a-8168-69c54559563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 56ec62dc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 56ec62dc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 56ec62dc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4262 (Wed Sep 26 17:40:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4262 (Wed Sep 26 17:40:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "01cda987", "local_conf_timestamp": 4262, "host-ts": 4262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Wed Sep 26 17:40:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4257 (Wed Sep 26 17:40:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ec6e945f", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false} lago.ssh: DEBUG: start task:12189e9a-68f8-42d0-ba7a-126282d28b98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:12189e9a-68f8-42d0-ba7a-126282d28b98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5aec01a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5aec01a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5aec01a8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4272 (Wed Sep 26 17:41:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4272 (Wed Sep 26 17:41:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03cc6058", "local_conf_timestamp": 4272, "host-ts": 4272}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Wed Sep 26 17:41:02 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4267 (Wed Sep 26 17:41:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e0a59550", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:6e63e48d-6a63-41da-92db-73b872e79e0a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6e63e48d-6a63-41da-92db-73b872e79e0a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5e4ddc0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5e4ddc0e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5e4ddc0e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4272 (Wed Sep 26 17:41:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4272 (Wed Sep 26 17:41:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03cc6058", "local_conf_timestamp": 4272, "host-ts": 4272}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Wed Sep 26 17:41:02 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4267 (Wed Sep 26 17:41:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e0a59550", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:b30d6158-ccd3-4752-8b2f-cad0c236e420:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b30d6158-ccd3-4752-8b2f-cad0c236e420:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6147daf4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6147daf4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6147daf4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4282 (Wed Sep 26 17:41:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4283 (Wed Sep 26 17:41:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f97e01d5", "local_conf_timestamp": 4283, "host-ts": 4282}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Wed Sep 26 17:41:12 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4277 (Wed Sep 26 17:41:12 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "93de5fcd", "local_conf_timestamp": 4277, "host-ts": 4277}, "global_maintenance": false} lago.ssh: DEBUG: start task:634c9638-320f-4848-ad7f-30ba5f612b08:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:634c9638-320f-4848-ad7f-30ba5f612b08:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6443f238 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6443f238 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6443f238 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4292 (Wed Sep 26 17:41:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4292 (Wed Sep 26 17:41:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "204a0f81", "local_conf_timestamp": 4292, "host-ts": 4292}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Wed Sep 26 17:41:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4287 (Wed Sep 26 17:41:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ac395afe", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false} lago.ssh: DEBUG: start task:e6c602be-6983-4556-98c4-4b57f63c7006:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e6c602be-6983-4556-98c4-4b57f63c7006:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 674f8cc6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 674f8cc6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 674f8cc6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4292 (Wed Sep 26 17:41:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4292 (Wed Sep 26 17:41:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "204a0f81", "local_conf_timestamp": 4292, "host-ts": 4292}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Wed Sep 26 17:41:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4287 (Wed Sep 26 17:41:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ac395afe", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false} lago.ssh: DEBUG: start task:4beec84d-e882-43d6-9fab-667e4e81cbda:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4beec84d-e882-43d6-9fab-667e4e81cbda:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6a680c80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6a680c80 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6a680c80 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4302 (Wed Sep 26 17:41:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4302 (Wed Sep 26 17:41:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8a734436", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4297 (Wed Sep 26 17:41:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4297 (Wed Sep 26 17:41:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a87fa5fb", "local_conf_timestamp": 4297, "host-ts": 4297}, "global_maintenance": false} lago.ssh: DEBUG: start task:7cd1cc45-58d8-4c69-92e3-62a2d08a6904:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7cd1cc45-58d8-4c69-92e3-62a2d08a6904:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6d5c6b66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6d5c6b66 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6d5c6b66 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4302 (Wed Sep 26 17:41:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4302 (Wed Sep 26 17:41:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8a734436", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4297 (Wed Sep 26 17:41:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4297 (Wed Sep 26 17:41:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a87fa5fb", "local_conf_timestamp": 4297, "host-ts": 4297}, "global_maintenance": false} lago.ssh: DEBUG: start task:a81b0359-b525-41fa-9906-7f21f0f27dd6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a81b0359-b525-41fa-9906-7f21f0f27dd6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 704b805a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 704b805a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 704b805a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4302 (Wed Sep 26 17:41:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4302 (Wed Sep 26 17:41:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8a734436", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4307 (Wed Sep 26 17:41:42 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4307 (Wed Sep 26 17:41:42 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4d44c828", "local_conf_timestamp": 4307, "host-ts": 4307}, "global_maintenance": false} lago.ssh: DEBUG: start task:e51f9ed4-1717-44c9-b0ee-37ea5e05676a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e51f9ed4-1717-44c9-b0ee-37ea5e05676a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7349afde on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7349afde on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7349afde on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4312 (Wed Sep 26 17:41:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4313 (Wed Sep 26 17:41:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "53474a62", "local_conf_timestamp": 4313, "host-ts": 4312}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4307 (Wed Sep 26 17:41:42 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4307 (Wed Sep 26 17:41:42 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4d44c828", "local_conf_timestamp": 4307, "host-ts": 4307}, "global_maintenance": false} lago.ssh: DEBUG: start task:243d2486-a3ca-4ca5-8fff-878978118a3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:243d2486-a3ca-4ca5-8fff-878978118a3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 76684004 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 76684004 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 76684004 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4322 (Wed Sep 26 17:41:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4322 (Wed Sep 26 17:41:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0d643ff3", "local_conf_timestamp": 4322, "host-ts": 4322}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4317 (Wed Sep 26 17:41:52 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4317 (Wed Sep 26 17:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4902372d", "local_conf_timestamp": 4317, "host-ts": 4317}, "global_maintenance": false} lago.ssh: DEBUG: start task:c1b8be35-3faa-4959-8a52-56991f4fa241:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1b8be35-3faa-4959-8a52-56991f4fa241:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7999ef0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7999ef0c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7999ef0c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4322 (Wed Sep 26 17:41:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4322 (Wed Sep 26 17:41:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0d643ff3", "local_conf_timestamp": 4322, "host-ts": 4322}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4317 (Wed Sep 26 17:41:52 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4317 (Wed Sep 26 17:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4902372d", "local_conf_timestamp": 4317, "host-ts": 4317}, "global_maintenance": false} lago.ssh: DEBUG: start task:c6947cc4-e42c-4461-a2ca-e144eed75f88:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c6947cc4-e42c-4461-a2ca-e144eed75f88:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7c7e5744 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7c7e5744 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7c7e5744 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4332 (Wed Sep 26 17:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4332 (Wed Sep 26 17:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c23b8814", "local_conf_timestamp": 4332, "host-ts": 4332}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4327 (Wed Sep 26 17:42:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4327 (Wed Sep 26 17:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4488c592", "local_conf_timestamp": 4327, "host-ts": 4327}, "global_maintenance": false} lago.ssh: DEBUG: start task:25677b7b-0a3f-4018-9a2d-399570c4f390:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:25677b7b-0a3f-4018-9a2d-399570c4f390:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7d97c89a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7d97c89a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7d97c89a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4332 (Wed Sep 26 17:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4332 (Wed Sep 26 17:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c23b8814", "local_conf_timestamp": 4332, "host-ts": 4332}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4327 (Wed Sep 26 17:42:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4327 (Wed Sep 26 17:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4488c592", "local_conf_timestamp": 4327, "host-ts": 4327}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:90555ac6-a749-4c6d-97d0-ca20f9ee7e56:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:90555ac6-a749-4c6d-97d0-ca20f9ee7e56:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 4d4a6eae on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4d4a6eae on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 4d4a6eae on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Wed Sep 26 17:40:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4241 (Wed Sep 26 17:40:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "49ba6ea7", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Wed Sep 26 17:40:41 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4247 (Wed Sep 26 17:40:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e8286b5a", "local_conf_timestamp": 4247, "host-ts": 4247}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cdd8bc8a-3f38-4e62-9699-7ef8e4864f9a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cdd8bc8a-3f38-4e62-9699-7ef8e4864f9a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4e65c50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 4e65c50e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:cdb61607-0d9d-4531-931e-e2a2c02f6b15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cdb61607-0d9d-4531-931e-e2a2c02f6b15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4fac57de on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4fac57de on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4fac57de on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Wed Sep 26 17:40:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4251 (Wed Sep 26 17:40:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "95dd2e98", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Wed Sep 26 17:40:41 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4247 (Wed Sep 26 17:40:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e8286b5a", "local_conf_timestamp": 4247, "host-ts": 4247}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1fc0139b-3d03-4e0a-8168-69c54559563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1fc0139b-3d03-4e0a-8168-69c54559563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 56ec62dc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 56ec62dc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 56ec62dc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4262 (Wed Sep 26 17:40:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4262 (Wed Sep 26 17:40:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "01cda987", "local_conf_timestamp": 4262, "host-ts": 4262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Wed Sep 26 17:40:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4257 (Wed Sep 26 17:40:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ec6e945f", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:12189e9a-68f8-42d0-ba7a-126282d28b98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:12189e9a-68f8-42d0-ba7a-126282d28b98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5aec01a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5aec01a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5aec01a8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4272 (Wed Sep 26 17:41:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4272 (Wed Sep 26 17:41:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03cc6058", "local_conf_timestamp": 4272, "host-ts": 4272}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Wed Sep 26 17:41:02 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4267 (Wed Sep 26 17:41:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e0a59550", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6e63e48d-6a63-41da-92db-73b872e79e0a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6e63e48d-6a63-41da-92db-73b872e79e0a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5e4ddc0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5e4ddc0e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5e4ddc0e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4272 (Wed Sep 26 17:41:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4272 (Wed Sep 26 17:41:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03cc6058", "local_conf_timestamp": 4272, "host-ts": 4272}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Wed Sep 26 17:41:02 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4267 (Wed Sep 26 17:41:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e0a59550", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b30d6158-ccd3-4752-8b2f-cad0c236e420:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b30d6158-ccd3-4752-8b2f-cad0c236e420:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6147daf4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6147daf4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6147daf4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4282 (Wed Sep 26 17:41:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4283 (Wed Sep 26 17:41:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f97e01d5", "local_conf_timestamp": 4283, "host-ts": 4282}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Wed Sep 26 17:41:12 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4277 (Wed Sep 26 17:41:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "93de5fcd", "local_conf_timestamp": 4277, "host-ts": 4277}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:634c9638-320f-4848-ad7f-30ba5f612b08:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:634c9638-320f-4848-ad7f-30ba5f612b08:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6443f238 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6443f238 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6443f238 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4292 (Wed Sep 26 17:41:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4292 (Wed Sep 26 17:41:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "204a0f81", "local_conf_timestamp": 4292, "host-ts": 4292}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Wed Sep 26 17:41:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4287 (Wed Sep 26 17:41:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ac395afe", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e6c602be-6983-4556-98c4-4b57f63c7006:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e6c602be-6983-4556-98c4-4b57f63c7006:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 674f8cc6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 674f8cc6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 674f8cc6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4292 (Wed Sep 26 17:41:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4292 (Wed Sep 26 17:41:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "204a0f81", "local_conf_timestamp": 4292, "host-ts": 4292}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Wed Sep 26 17:41:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4287 (Wed Sep 26 17:41:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ac395afe", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4beec84d-e882-43d6-9fab-667e4e81cbda:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4beec84d-e882-43d6-9fab-667e4e81cbda:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6a680c80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6a680c80 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6a680c80 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4302 (Wed Sep 26 17:41:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4302 (Wed Sep 26 17:41:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8a734436", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4297 (Wed Sep 26 17:41:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4297 (Wed Sep 26 17:41:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a87fa5fb", "local_conf_timestamp": 4297, "host-ts": 4297}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7cd1cc45-58d8-4c69-92e3-62a2d08a6904:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7cd1cc45-58d8-4c69-92e3-62a2d08a6904:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6d5c6b66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6d5c6b66 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6d5c6b66 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4302 (Wed Sep 26 17:41:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4302 (Wed Sep 26 17:41:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8a734436", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4297 (Wed Sep 26 17:41:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4297 (Wed Sep 26 17:41:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a87fa5fb", "local_conf_timestamp": 4297, "host-ts": 4297}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a81b0359-b525-41fa-9906-7f21f0f27dd6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a81b0359-b525-41fa-9906-7f21f0f27dd6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 704b805a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 704b805a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 704b805a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4302 (Wed Sep 26 17:41:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4302 (Wed Sep 26 17:41:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8a734436", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4307 (Wed Sep 26 17:41:42 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4307 (Wed Sep 26 17:41:42 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4d44c828", "local_conf_timestamp": 4307, "host-ts": 4307}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e51f9ed4-1717-44c9-b0ee-37ea5e05676a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e51f9ed4-1717-44c9-b0ee-37ea5e05676a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7349afde on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7349afde on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7349afde on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4312 (Wed Sep 26 17:41:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4313 (Wed Sep 26 17:41:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "53474a62", "local_conf_timestamp": 4313, "host-ts": 4312}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4307 (Wed Sep 26 17:41:42 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4307 (Wed Sep 26 17:41:42 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4d44c828", "local_conf_timestamp": 4307, "host-ts": 4307}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:243d2486-a3ca-4ca5-8fff-878978118a3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:243d2486-a3ca-4ca5-8fff-878978118a3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 76684004 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 76684004 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 76684004 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4322 (Wed Sep 26 17:41:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4322 (Wed Sep 26 17:41:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0d643ff3", "local_conf_timestamp": 4322, "host-ts": 4322}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4317 (Wed Sep 26 17:41:52 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4317 (Wed Sep 26 17:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4902372d", "local_conf_timestamp": 4317, "host-ts": 4317}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c1b8be35-3faa-4959-8a52-56991f4fa241:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1b8be35-3faa-4959-8a52-56991f4fa241:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7999ef0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7999ef0c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7999ef0c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4322 (Wed Sep 26 17:41:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4322 (Wed Sep 26 17:41:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0d643ff3", "local_conf_timestamp": 4322, "host-ts": 4322}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4317 (Wed Sep 26 17:41:52 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4317 (Wed Sep 26 17:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4902372d", "local_conf_timestamp": 4317, "host-ts": 4317}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c6947cc4-e42c-4461-a2ca-e144eed75f88:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c6947cc4-e42c-4461-a2ca-e144eed75f88:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7c7e5744 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7c7e5744 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7c7e5744 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4332 (Wed Sep 26 17:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4332 (Wed Sep 26 17:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c23b8814", "local_conf_timestamp": 4332, "host-ts": 4332}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4327 (Wed Sep 26 17:42:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4327 (Wed Sep 26 17:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4488c592", "local_conf_timestamp": 4327, "host-ts": 4327}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:25677b7b-0a3f-4018-9a2d-399570c4f390:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:25677b7b-0a3f-4018-9a2d-399570c4f390:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7d97c89a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7d97c89a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7d97c89a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4332 (Wed Sep 26 17:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4332 (Wed Sep 26 17:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c23b8814", "local_conf_timestamp": 4332, "host-ts": 4332}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4327 (Wed Sep 26 17:42:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4327 (Wed Sep 26 17:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4488c592", "local_conf_timestamp": 4327, "host-ts": 4327}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/348/ Build Number: 348 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/349/ Build Number: 349 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/350/ Build Number: 350 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:ea8e18b3-36bd-4a60-ab82-3517ab01b5ef:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:ea8e18b3-36bd-4a60-ab82-3517ab01b5ef:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 309a6f6a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 309a6f6a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 309a6f6a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4200 (Thu Sep 27 17:38:38 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4200 (Thu Sep 27 17:38:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3f8892f7", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Thu Sep 27 17:38:41 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4204 (Thu Sep 27 17:38:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7da04a36", "local_conf_timestamp": 4204, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:05e8727e-adc6-47ef-88d6-970c7d4213d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:05e8727e-adc6-47ef-88d6-970c7d4213d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 31bfdc5e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 31bfdc5e on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:57d79342-2d7e-4822-8c43-2c2be7baa29b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:57d79342-2d7e-4822-8c43-2c2be7baa29b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 32f71678 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 32f71678 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 32f71678 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Thu Sep 27 17:38:48 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4211 (Thu Sep 27 17:38:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94e8e25e", "local_conf_timestamp": 4211, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Thu Sep 27 17:38:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4213 (Thu Sep 27 17:38:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7822090", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:451ab0e3-a3ad-4327-98c7-0f3d4a6901be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:451ab0e3-a3ad-4327-98c7-0f3d4a6901be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3a3cbde8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3a3cbde8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3a3cbde8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Thu Sep 27 17:38:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4221 (Thu Sep 27 17:38:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ce15b3ca", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Thu Sep 27 17:38:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4213 (Thu Sep 27 17:38:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7822090", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:c3f59c1f-142c-4454-b0cf-b280b9b971de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c3f59c1f-142c-4454-b0cf-b280b9b971de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3d74ee68 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3d74ee68 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3d74ee68 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Thu Sep 27 17:39:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4231 (Thu Sep 27 17:39:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Thu Sep 27 17:39:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4223 (Thu Sep 27 17:39:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:8f88ee3b-81bf-49e4-bc0c-2343c22d0ef4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8f88ee3b-81bf-49e4-bc0c-2343c22d0ef4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 408b4750 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 408b4750 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 408b4750 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Thu Sep 27 17:39:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4231 (Thu Sep 27 17:39:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Thu Sep 27 17:39:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4223 (Thu Sep 27 17:39:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:a5855a1b-28b3-444e-8be2-5b3f6b06f6df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a5855a1b-28b3-444e-8be2-5b3f6b06f6df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 436c66c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 436c66c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 436c66c0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Thu Sep 27 17:39:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4241 (Thu Sep 27 17:39:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d764ba26", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Thu Sep 27 17:39:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4233 (Thu Sep 27 17:39:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9832eb02", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:3e1c18fd-f5b5-4e41-885d-d93dbc9d4b40:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3e1c18fd-f5b5-4e41-885d-d93dbc9d4b40:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 465a2598 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 465a2598 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 465a2598 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Thu Sep 27 17:39:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4241 (Thu Sep 27 17:39:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d764ba26", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Thu Sep 27 17:39:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4233 (Thu Sep 27 17:39:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9832eb02", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:2d1c0253-6d87-4b6b-b1e6-5f5e93f45d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2d1c0253-6d87-4b6b-b1e6-5f5e93f45d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 496c4c16 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 496c4c16 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 496c4c16 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Thu Sep 27 17:39:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4251 (Thu Sep 27 17:39:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "795784e4", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Thu Sep 27 17:39:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4244 (Thu Sep 27 17:39:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b8683ba", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false} lago.ssh: DEBUG: start task:27029167-3baa-42a6-9ab5-39da5a65ae13:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:27029167-3baa-42a6-9ab5-39da5a65ae13:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4c73eb12 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4c73eb12 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4c73eb12 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Thu Sep 27 17:39:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4251 (Thu Sep 27 17:39:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "795784e4", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Thu Sep 27 17:39:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4244 (Thu Sep 27 17:39:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b8683ba", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false} lago.ssh: DEBUG: start task:f837bcfc-cacd-494a-a35f-5c1ccde6e4ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f837bcfc-cacd-494a-a35f-5c1ccde6e4ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4f633512 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4f633512 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4f633512 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4261 (Thu Sep 27 17:39:38 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4261 (Thu Sep 27 17:39:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5073c1e3", "local_conf_timestamp": 4261, "host-ts": 4261}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Thu Sep 27 17:39:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4253 (Thu Sep 27 17:39:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81a4e91c", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false} lago.ssh: DEBUG: start task:389198af-4e4d-4a33-acd7-109c009ae20d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:389198af-4e4d-4a33-acd7-109c009ae20d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 52744c82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 52744c82 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 52744c82 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4261 (Thu Sep 27 17:39:38 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4261 (Thu Sep 27 17:39:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5073c1e3", "local_conf_timestamp": 4261, "host-ts": 4261}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Thu Sep 27 17:39:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4253 (Thu Sep 27 17:39:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81a4e91c", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false} lago.ssh: DEBUG: start task:b749cce2-a1bb-4a90-aff8-5d8213242c04:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b749cce2-a1bb-4a90-aff8-5d8213242c04:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 55a3cca2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 55a3cca2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 55a3cca2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4271 (Thu Sep 27 17:39:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4271 (Thu Sep 27 17:39:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fe40ff21", "local_conf_timestamp": 4271, "host-ts": 4271}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Thu Sep 27 17:39:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4263 (Thu Sep 27 17:39:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3e4b309d", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false} lago.ssh: DEBUG: start task:7315593a-ac15-4a08-bf66-3bf8b3371c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7315593a-ac15-4a08-bf66-3bf8b3371c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 58e08522 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 58e08522 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 58e08522 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4271 (Thu Sep 27 17:39:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4271 (Thu Sep 27 17:39:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fe40ff21", "local_conf_timestamp": 4271, "host-ts": 4271}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Thu Sep 27 17:39:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4263 (Thu Sep 27 17:39:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3e4b309d", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false} lago.ssh: DEBUG: start task:668a7844-5ee0-42e2-bb6b-79daf7e20562:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:668a7844-5ee0-42e2-bb6b-79daf7e20562:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5bd7d118 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5bd7d118 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5bd7d118 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4281 (Thu Sep 27 17:39:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4281 (Thu Sep 27 17:39:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "73f5ae3a", "local_conf_timestamp": 4281, "host-ts": 4281}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Thu Sep 27 17:39:51 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4273 (Thu Sep 27 17:39:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3a0dcf98", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false} lago.ssh: DEBUG: start task:8b2c4f71-066d-4f17-ba3a-96ea07db9286:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8b2c4f71-066d-4f17-ba3a-96ea07db9286:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5efdd5d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5efdd5d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5efdd5d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4281 (Thu Sep 27 17:39:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4281 (Thu Sep 27 17:39:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "73f5ae3a", "local_conf_timestamp": 4281, "host-ts": 4281}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Thu Sep 27 17:39:51 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4273 (Thu Sep 27 17:39:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3a0dcf98", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false} lago.ssh: DEBUG: start task:aecfb02e-adb2-45c1-84c8-0fae10f68591:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aecfb02e-adb2-45c1-84c8-0fae10f68591:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 61de6b94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 61de6b94 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 61de6b94 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4291 (Thu Sep 27 17:40:09 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4292 (Thu Sep 27 17:40:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3dcfc5c1", "local_conf_timestamp": 4292, "host-ts": 4291}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4283 (Thu Sep 27 17:40:01 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4284 (Thu Sep 27 17:40:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false} lago.ssh: DEBUG: start task:c9ee20ad-6b4d-440b-b317-73217d37dd31:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c9ee20ad-6b4d-440b-b317-73217d37dd31:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 630555aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 630555aa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 630555aa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4291 (Thu Sep 27 17:40:09 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4292 (Thu Sep 27 17:40:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3dcfc5c1", "local_conf_timestamp": 4292, "host-ts": 4291}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4283 (Thu Sep 27 17:40:01 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4284 (Thu Sep 27 17:40:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false} lago.ssh: DEBUG: start task:5f006cc1-89ad-43dd-89ac-4180169375ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5f006cc1-89ad-43dd-89ac-4180169375ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6a175294 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6a175294 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6a175294 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4302 (Thu Sep 27 17:40:20 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4302 (Thu Sep 27 17:40:20 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "f7f7f6b4", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4293 (Thu Sep 27 17:40:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4293 (Thu Sep 27 17:40:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b0c19d29", "local_conf_timestamp": 4293, "host-ts": 4293}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:d5de2191-b962-4f3a-b898-587010be47f4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d5de2191-b962-4f3a-b898-587010be47f4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1dfc412a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none lago.ssh: DEBUG: Command 1dfc412a on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for Engine Maintenance to reset... lago.ssh: DEBUG: start task:0a270c25-c332-41b1-9b68-ea72b3a1d580:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0a270c25-c332-41b1-9b68-ea72b3a1d580:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 20385208 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 20385208 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 20385208 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4603 (Thu Sep 27 17:45:21 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4603 (Thu Sep 27 17:45:21 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "30e65eec", "local_conf_timestamp": 4603, "host-ts": 4603}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4604 (Thu Sep 27 17:45:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4604 (Thu Sep 27 17:45:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2e503cd4", "local_conf_timestamp": 4604, "host-ts": 4604}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 130, in local_maintenance _wait_for_engine_maintenance(host, he_index, False) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 69, in _wait_for_engine_maintenance testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:ea8e18b3-36bd-4a60-ab82-3517ab01b5ef:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:ea8e18b3-36bd-4a60-ab82-3517ab01b5ef:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 309a6f6a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 309a6f6a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 309a6f6a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4200 (Thu Sep 27 17:38:38 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4200 (Thu Sep 27 17:38:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3f8892f7", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Thu Sep 27 17:38:41 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4204 (Thu Sep 27 17:38:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7da04a36", "local_conf_timestamp": 4204, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:05e8727e-adc6-47ef-88d6-970c7d4213d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:05e8727e-adc6-47ef-88d6-970c7d4213d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 31bfdc5e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 31bfdc5e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:57d79342-2d7e-4822-8c43-2c2be7baa29b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:57d79342-2d7e-4822-8c43-2c2be7baa29b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 32f71678 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 32f71678 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 32f71678 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Thu Sep 27 17:38:48 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4211 (Thu Sep 27 17:38:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94e8e25e", "local_conf_timestamp": 4211, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Thu Sep 27 17:38:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4213 (Thu Sep 27 17:38:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7822090", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:451ab0e3-a3ad-4327-98c7-0f3d4a6901be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:451ab0e3-a3ad-4327-98c7-0f3d4a6901be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3a3cbde8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3a3cbde8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3a3cbde8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Thu Sep 27 17:38:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4221 (Thu Sep 27 17:38:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ce15b3ca", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Thu Sep 27 17:38:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4213 (Thu Sep 27 17:38:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7822090", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c3f59c1f-142c-4454-b0cf-b280b9b971de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c3f59c1f-142c-4454-b0cf-b280b9b971de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3d74ee68 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3d74ee68 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3d74ee68 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Thu Sep 27 17:39:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4231 (Thu Sep 27 17:39:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Thu Sep 27 17:39:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4223 (Thu Sep 27 17:39:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8f88ee3b-81bf-49e4-bc0c-2343c22d0ef4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8f88ee3b-81bf-49e4-bc0c-2343c22d0ef4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 408b4750 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 408b4750 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 408b4750 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Thu Sep 27 17:39:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4231 (Thu Sep 27 17:39:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Thu Sep 27 17:39:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4223 (Thu Sep 27 17:39:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a5855a1b-28b3-444e-8be2-5b3f6b06f6df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a5855a1b-28b3-444e-8be2-5b3f6b06f6df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 436c66c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 436c66c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 436c66c0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Thu Sep 27 17:39:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4241 (Thu Sep 27 17:39:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d764ba26", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Thu Sep 27 17:39:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4233 (Thu Sep 27 17:39:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9832eb02", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3e1c18fd-f5b5-4e41-885d-d93dbc9d4b40:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3e1c18fd-f5b5-4e41-885d-d93dbc9d4b40:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 465a2598 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 465a2598 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 465a2598 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Thu Sep 27 17:39:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4241 (Thu Sep 27 17:39:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d764ba26", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Thu Sep 27 17:39:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4233 (Thu Sep 27 17:39:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9832eb02", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2d1c0253-6d87-4b6b-b1e6-5f5e93f45d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2d1c0253-6d87-4b6b-b1e6-5f5e93f45d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 496c4c16 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 496c4c16 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 496c4c16 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Thu Sep 27 17:39:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4251 (Thu Sep 27 17:39:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "795784e4", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Thu Sep 27 17:39:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4244 (Thu Sep 27 17:39:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b8683ba", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:27029167-3baa-42a6-9ab5-39da5a65ae13:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:27029167-3baa-42a6-9ab5-39da5a65ae13:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4c73eb12 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4c73eb12 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4c73eb12 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Thu Sep 27 17:39:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4251 (Thu Sep 27 17:39:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "795784e4", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Thu Sep 27 17:39:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4244 (Thu Sep 27 17:39:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b8683ba", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f837bcfc-cacd-494a-a35f-5c1ccde6e4ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f837bcfc-cacd-494a-a35f-5c1ccde6e4ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4f633512 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4f633512 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4f633512 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4261 (Thu Sep 27 17:39:38 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4261 (Thu Sep 27 17:39:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5073c1e3", "local_conf_timestamp": 4261, "host-ts": 4261}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Thu Sep 27 17:39:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4253 (Thu Sep 27 17:39:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81a4e91c", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:389198af-4e4d-4a33-acd7-109c009ae20d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:389198af-4e4d-4a33-acd7-109c009ae20d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 52744c82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 52744c82 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 52744c82 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4261 (Thu Sep 27 17:39:38 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4261 (Thu Sep 27 17:39:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5073c1e3", "local_conf_timestamp": 4261, "host-ts": 4261}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Thu Sep 27 17:39:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4253 (Thu Sep 27 17:39:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81a4e91c", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b749cce2-a1bb-4a90-aff8-5d8213242c04:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b749cce2-a1bb-4a90-aff8-5d8213242c04:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 55a3cca2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 55a3cca2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 55a3cca2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4271 (Thu Sep 27 17:39:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4271 (Thu Sep 27 17:39:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fe40ff21", "local_conf_timestamp": 4271, "host-ts": 4271}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Thu Sep 27 17:39:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4263 (Thu Sep 27 17:39:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3e4b309d", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7315593a-ac15-4a08-bf66-3bf8b3371c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7315593a-ac15-4a08-bf66-3bf8b3371c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 58e08522 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 58e08522 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 58e08522 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4271 (Thu Sep 27 17:39:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4271 (Thu Sep 27 17:39:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fe40ff21", "local_conf_timestamp": 4271, "host-ts": 4271}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Thu Sep 27 17:39:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4263 (Thu Sep 27 17:39:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3e4b309d", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:668a7844-5ee0-42e2-bb6b-79daf7e20562:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:668a7844-5ee0-42e2-bb6b-79daf7e20562:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5bd7d118 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5bd7d118 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5bd7d118 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4281 (Thu Sep 27 17:39:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4281 (Thu Sep 27 17:39:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "73f5ae3a", "local_conf_timestamp": 4281, "host-ts": 4281}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Thu Sep 27 17:39:51 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4273 (Thu Sep 27 17:39:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3a0dcf98", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8b2c4f71-066d-4f17-ba3a-96ea07db9286:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8b2c4f71-066d-4f17-ba3a-96ea07db9286:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5efdd5d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5efdd5d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5efdd5d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4281 (Thu Sep 27 17:39:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4281 (Thu Sep 27 17:39:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "73f5ae3a", "local_conf_timestamp": 4281, "host-ts": 4281}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Thu Sep 27 17:39:51 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4273 (Thu Sep 27 17:39:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3a0dcf98", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aecfb02e-adb2-45c1-84c8-0fae10f68591:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aecfb02e-adb2-45c1-84c8-0fae10f68591:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 61de6b94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 61de6b94 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 61de6b94 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4291 (Thu Sep 27 17:40:09 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4292 (Thu Sep 27 17:40:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3dcfc5c1", "local_conf_timestamp": 4292, "host-ts": 4291}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4283 (Thu Sep 27 17:40:01 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4284 (Thu Sep 27 17:40:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c9ee20ad-6b4d-440b-b317-73217d37dd31:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c9ee20ad-6b4d-440b-b317-73217d37dd31:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 630555aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 630555aa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 630555aa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4291 (Thu Sep 27 17:40:09 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4292 (Thu Sep 27 17:40:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3dcfc5c1", "local_conf_timestamp": 4292, "host-ts": 4291}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4283 (Thu Sep 27 17:40:01 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4284 (Thu Sep 27 17:40:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5f006cc1-89ad-43dd-89ac-4180169375ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5f006cc1-89ad-43dd-89ac-4180169375ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6a175294 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6a175294 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6a175294 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4302 (Thu Sep 27 17:40:20 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4302 (Thu Sep 27 17:40:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "f7f7f6b4", "local_conf_timestamp": 4302, "host-ts": 4302}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4293 (Thu Sep 27 17:40:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4293 (Thu Sep 27 17:40:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b0c19d29", "local_conf_timestamp": 4293, "host-ts": 4293}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:d5de2191-b962-4f3a-b898-587010be47f4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d5de2191-b962-4f3a-b898-587010be47f4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1dfc412a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none\nlago.ssh: DEBUG: Command 1dfc412a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for Engine Maintenance to reset...\nlago.ssh: DEBUG: start task:0a270c25-c332-41b1-9b68-ea72b3a1d580:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0a270c25-c332-41b1-9b68-ea72b3a1d580:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 20385208 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 20385208 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 20385208 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4603 (Thu Sep 27 17:45:21 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4603 (Thu Sep 27 17:45:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "30e65eec", "local_conf_timestamp": 4603, "host-ts": 4603}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4604 (Thu Sep 27 17:45:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4604 (Thu Sep 27 17:45:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2e503cd4", "local_conf_timestamp": 4604, "host-ts": 4604}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/351/ Build Number: 351 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:67141e8e-e5d8-43dd-9a0b-8808087cc83d:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:67141e8e-e5d8-43dd-9a0b-8808087cc83d:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running dfad6316 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dfad6316 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command dfad6316 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Fri Sep 28 01:43:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4184 (Fri Sep 28 01:43:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c21c9ef7", "local_conf_timestamp": 4184, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Fri Sep 28 01:43:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4178 (Fri Sep 28 01:43:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "26535263", "local_conf_timestamp": 4178, "host-ts": 4178}, "global_maintenance": false} lago.ssh: DEBUG: start task:2be9b263-1ed0-479e-b5ae-01cccfdbd08a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2be9b263-1ed0-479e-b5ae-01cccfdbd08a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e0bacac8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command e0bacac8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:3e135bd6-9467-4866-a5d1-f8f580a800f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3e135bd6-9467-4866-a5d1-f8f580a800f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e1d4f5a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e1d4f5a0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e1d4f5a0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Fri Sep 28 01:43:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4184 (Fri Sep 28 01:43:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c21c9ef7", "local_conf_timestamp": 4184, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Fri Sep 28 01:43:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4189 (Fri Sep 28 01:43:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6eb367c6", "local_conf_timestamp": 4189, "host-ts": 4188}, "global_maintenance": false} lago.ssh: DEBUG: start task:60b75262-e3bc-411d-9012-d98b51c42703:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:60b75262-e3bc-411d-9012-d98b51c42703:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e8e261a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e8e261a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e8e261a2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4194 (Fri Sep 28 01:43:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4194 (Fri Sep 28 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d1489e4", "local_conf_timestamp": 4194, "host-ts": 4194}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Fri Sep 28 01:43:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4189 (Fri Sep 28 01:43:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6eb367c6", "local_conf_timestamp": 4189, "host-ts": 4188}, "global_maintenance": false} lago.ssh: DEBUG: start task:33a74bf1-cd01-4196-877d-87270b2124e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:33a74bf1-cd01-4196-877d-87270b2124e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ec5f9eb2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ec5f9eb2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ec5f9eb2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4204 (Fri Sep 28 01:43:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4204 (Fri Sep 28 01:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dfdfec27", "local_conf_timestamp": 4204, "host-ts": 4204}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Fri Sep 28 01:43:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4199 (Fri Sep 28 01:43:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "802bb8d3", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:c083c2fe-596d-49fa-802d-4d16e301380e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c083c2fe-596d-49fa-802d-4d16e301380e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ef496fa4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ef496fa4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ef496fa4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4214 (Fri Sep 28 01:43:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4214 (Fri Sep 28 01:43:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Fri Sep 28 01:43:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4209 (Fri Sep 28 01:43:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41d45cf6", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:9ca1dc68-5cd9-4cb1-a659-f32cd355bf0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9ca1dc68-5cd9-4cb1-a659-f32cd355bf0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f2501ba8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f2501ba8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f2501ba8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4214 (Fri Sep 28 01:43:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4214 (Fri Sep 28 01:43:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Fri Sep 28 01:43:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4209 (Fri Sep 28 01:43:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41d45cf6", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:3a0e8b7c-d6fc-45a9-828c-05dd3d6ca47d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3a0e8b7c-d6fc-45a9-828c-05dd3d6ca47d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f536fe72 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f536fe72 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f536fe72 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Fri Sep 28 01:43:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Fri Sep 28 01:43:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2fcfa774", "local_conf_timestamp": 4225, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Fri Sep 28 01:43:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4219 (Fri Sep 28 01:43:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4592a3f3", "local_conf_timestamp": 4219, "host-ts": 4219}, "global_maintenance": false} lago.ssh: DEBUG: start task:b757597c-86e3-4703-b5e9-1b7fc703ff7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b757597c-86e3-4703-b5e9-1b7fc703ff7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f89d0b74 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f89d0b74 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f89d0b74 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Fri Sep 28 01:43:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Fri Sep 28 01:43:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2fcfa774", "local_conf_timestamp": 4225, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Fri Sep 28 01:43:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4219 (Fri Sep 28 01:43:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4592a3f3", "local_conf_timestamp": 4219, "host-ts": 4219}, "global_maintenance": false} lago.ssh: DEBUG: start task:5f6a31d2-df74-4f2a-b868-60ae5bdcb9d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5f6a31d2-df74-4f2a-b868-60ae5bdcb9d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fb952578 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fb952578 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fb952578 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Fri Sep 28 01:44:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4234 (Fri Sep 28 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Fri Sep 28 01:44:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4230 (Fri Sep 28 01:44:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9abaf397", "local_conf_timestamp": 4230, "host-ts": 4229}, "global_maintenance": false} lago.ssh: DEBUG: start task:6e991f25-341b-4441-8031-076ba4ddbd28:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6e991f25-341b-4441-8031-076ba4ddbd28:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fe8e94d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fe8e94d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fe8e94d0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Fri Sep 28 01:44:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4234 (Fri Sep 28 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Fri Sep 28 01:44:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4230 (Fri Sep 28 01:44:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9abaf397", "local_conf_timestamp": 4230, "host-ts": 4229}, "global_maintenance": false} lago.ssh: DEBUG: start task:4af3eb97-7b6b-484e-a81a-107bf5c18788:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4af3eb97-7b6b-484e-a81a-107bf5c18788:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0199c65e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0199c65e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0199c65e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Fri Sep 28 01:44:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4245 (Fri Sep 28 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d872d7a", "local_conf_timestamp": 4245, "host-ts": 4244}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Fri Sep 28 01:44:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4240 (Fri Sep 28 01:44:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:96df43df-3dd6-4d8f-bee8-231239ceb6b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:96df43df-3dd6-4d8f-bee8-231239ceb6b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 04bd61c4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 04bd61c4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 04bd61c4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Fri Sep 28 01:44:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4245 (Fri Sep 28 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d872d7a", "local_conf_timestamp": 4245, "host-ts": 4244}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Fri Sep 28 01:44:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4240 (Fri Sep 28 01:44:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:d7d5d273-a30e-4105-888a-0ca7aee4a886:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d7d5d273-a30e-4105-888a-0ca7aee4a886:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 07ada196 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 07ada196 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 07ada196 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Fri Sep 28 01:44:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4255 (Fri Sep 28 01:44:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "46e7abc6", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Fri Sep 28 01:44:20 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4250 (Fri Sep 28 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:03697026-035b-4d24-afc6-f52b7c95d881:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:03697026-035b-4d24-afc6-f52b7c95d881:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0abaa352 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0abaa352 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0abaa352 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Fri Sep 28 01:44:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4255 (Fri Sep 28 01:44:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "46e7abc6", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Fri Sep 28 01:44:20 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4250 (Fri Sep 28 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:d12e3bd4-3e9f-4b5d-80bb-f6184e8625ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d12e3bd4-3e9f-4b5d-80bb-f6184e8625ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0d968938 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0d968938 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0d968938 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Fri Sep 28 01:44:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4255 (Fri Sep 28 01:44:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "46e7abc6", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Fri Sep 28 01:44:20 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4250 (Fri Sep 28 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:ea24609e-40ad-4166-a593-f19cd1571e95:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ea24609e-40ad-4166-a593-f19cd1571e95:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1072eb88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1072eb88 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1072eb88 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4265 (Fri Sep 28 01:44:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4266 (Fri Sep 28 01:44:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18df8829", "local_conf_timestamp": 4266, "host-ts": 4265}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4261 (Fri Sep 28 01:44:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4260 (Fri Sep 28 01:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "53049f61", "local_conf_timestamp": 4260, "host-ts": 4261}, "global_maintenance": false} lago.ssh: DEBUG: start task:9c3afeba-490b-4319-ba90-513761ea340d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9c3afeba-490b-4319-ba90-513761ea340d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 11820e32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 11820e32 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 11820e32 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4265 (Fri Sep 28 01:44:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4266 (Fri Sep 28 01:44:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18df8829", "local_conf_timestamp": 4266, "host-ts": 4265}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4261 (Fri Sep 28 01:44:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4260 (Fri Sep 28 01:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "53049f61", "local_conf_timestamp": 4260, "host-ts": 4261}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:67141e8e-e5d8-43dd-9a0b-8808087cc83d:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:67141e8e-e5d8-43dd-9a0b-8808087cc83d:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running dfad6316 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dfad6316 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command dfad6316 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Fri Sep 28 01:43:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4184 (Fri Sep 28 01:43:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c21c9ef7", "local_conf_timestamp": 4184, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Fri Sep 28 01:43:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4178 (Fri Sep 28 01:43:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "26535263", "local_conf_timestamp": 4178, "host-ts": 4178}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2be9b263-1ed0-479e-b5ae-01cccfdbd08a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2be9b263-1ed0-479e-b5ae-01cccfdbd08a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e0bacac8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command e0bacac8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:3e135bd6-9467-4866-a5d1-f8f580a800f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3e135bd6-9467-4866-a5d1-f8f580a800f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e1d4f5a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e1d4f5a0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e1d4f5a0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Fri Sep 28 01:43:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4184 (Fri Sep 28 01:43:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c21c9ef7", "local_conf_timestamp": 4184, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Fri Sep 28 01:43:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4189 (Fri Sep 28 01:43:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6eb367c6", "local_conf_timestamp": 4189, "host-ts": 4188}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:60b75262-e3bc-411d-9012-d98b51c42703:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:60b75262-e3bc-411d-9012-d98b51c42703:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e8e261a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e8e261a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e8e261a2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4194 (Fri Sep 28 01:43:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4194 (Fri Sep 28 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d1489e4", "local_conf_timestamp": 4194, "host-ts": 4194}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Fri Sep 28 01:43:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4189 (Fri Sep 28 01:43:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6eb367c6", "local_conf_timestamp": 4189, "host-ts": 4188}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:33a74bf1-cd01-4196-877d-87270b2124e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:33a74bf1-cd01-4196-877d-87270b2124e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ec5f9eb2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ec5f9eb2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ec5f9eb2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4204 (Fri Sep 28 01:43:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4204 (Fri Sep 28 01:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dfdfec27", "local_conf_timestamp": 4204, "host-ts": 4204}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Fri Sep 28 01:43:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4199 (Fri Sep 28 01:43:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "802bb8d3", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c083c2fe-596d-49fa-802d-4d16e301380e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c083c2fe-596d-49fa-802d-4d16e301380e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ef496fa4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ef496fa4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ef496fa4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4214 (Fri Sep 28 01:43:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4214 (Fri Sep 28 01:43:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Fri Sep 28 01:43:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4209 (Fri Sep 28 01:43:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41d45cf6", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9ca1dc68-5cd9-4cb1-a659-f32cd355bf0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9ca1dc68-5cd9-4cb1-a659-f32cd355bf0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f2501ba8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f2501ba8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f2501ba8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4214 (Fri Sep 28 01:43:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4214 (Fri Sep 28 01:43:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Fri Sep 28 01:43:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4209 (Fri Sep 28 01:43:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41d45cf6", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3a0e8b7c-d6fc-45a9-828c-05dd3d6ca47d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3a0e8b7c-d6fc-45a9-828c-05dd3d6ca47d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f536fe72 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f536fe72 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f536fe72 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Fri Sep 28 01:43:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Fri Sep 28 01:43:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2fcfa774", "local_conf_timestamp": 4225, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Fri Sep 28 01:43:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4219 (Fri Sep 28 01:43:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4592a3f3", "local_conf_timestamp": 4219, "host-ts": 4219}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b757597c-86e3-4703-b5e9-1b7fc703ff7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b757597c-86e3-4703-b5e9-1b7fc703ff7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f89d0b74 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f89d0b74 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f89d0b74 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Fri Sep 28 01:43:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Fri Sep 28 01:43:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2fcfa774", "local_conf_timestamp": 4225, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Fri Sep 28 01:43:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4219 (Fri Sep 28 01:43:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4592a3f3", "local_conf_timestamp": 4219, "host-ts": 4219}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5f6a31d2-df74-4f2a-b868-60ae5bdcb9d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5f6a31d2-df74-4f2a-b868-60ae5bdcb9d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fb952578 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fb952578 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fb952578 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Fri Sep 28 01:44:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4234 (Fri Sep 28 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Fri Sep 28 01:44:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4230 (Fri Sep 28 01:44:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9abaf397", "local_conf_timestamp": 4230, "host-ts": 4229}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6e991f25-341b-4441-8031-076ba4ddbd28:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6e991f25-341b-4441-8031-076ba4ddbd28:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fe8e94d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fe8e94d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fe8e94d0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Fri Sep 28 01:44:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4234 (Fri Sep 28 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Fri Sep 28 01:44:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4230 (Fri Sep 28 01:44:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9abaf397", "local_conf_timestamp": 4230, "host-ts": 4229}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4af3eb97-7b6b-484e-a81a-107bf5c18788:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4af3eb97-7b6b-484e-a81a-107bf5c18788:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0199c65e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0199c65e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0199c65e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Fri Sep 28 01:44:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4245 (Fri Sep 28 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d872d7a", "local_conf_timestamp": 4245, "host-ts": 4244}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Fri Sep 28 01:44:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4240 (Fri Sep 28 01:44:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:96df43df-3dd6-4d8f-bee8-231239ceb6b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:96df43df-3dd6-4d8f-bee8-231239ceb6b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 04bd61c4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 04bd61c4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 04bd61c4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Fri Sep 28 01:44:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4245 (Fri Sep 28 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d872d7a", "local_conf_timestamp": 4245, "host-ts": 4244}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Fri Sep 28 01:44:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4240 (Fri Sep 28 01:44:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d7d5d273-a30e-4105-888a-0ca7aee4a886:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d7d5d273-a30e-4105-888a-0ca7aee4a886:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 07ada196 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 07ada196 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 07ada196 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Fri Sep 28 01:44:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4255 (Fri Sep 28 01:44:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "46e7abc6", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Fri Sep 28 01:44:20 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4250 (Fri Sep 28 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:03697026-035b-4d24-afc6-f52b7c95d881:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:03697026-035b-4d24-afc6-f52b7c95d881:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0abaa352 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0abaa352 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0abaa352 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Fri Sep 28 01:44:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4255 (Fri Sep 28 01:44:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "46e7abc6", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Fri Sep 28 01:44:20 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4250 (Fri Sep 28 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d12e3bd4-3e9f-4b5d-80bb-f6184e8625ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d12e3bd4-3e9f-4b5d-80bb-f6184e8625ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0d968938 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0d968938 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0d968938 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Fri Sep 28 01:44:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4255 (Fri Sep 28 01:44:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "46e7abc6", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Fri Sep 28 01:44:20 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4250 (Fri Sep 28 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ea24609e-40ad-4166-a593-f19cd1571e95:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ea24609e-40ad-4166-a593-f19cd1571e95:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1072eb88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1072eb88 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1072eb88 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4265 (Fri Sep 28 01:44:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4266 (Fri Sep 28 01:44:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18df8829", "local_conf_timestamp": 4266, "host-ts": 4265}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4261 (Fri Sep 28 01:44:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4260 (Fri Sep 28 01:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "53049f61", "local_conf_timestamp": 4260, "host-ts": 4261}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9c3afeba-490b-4319-ba90-513761ea340d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9c3afeba-490b-4319-ba90-513761ea340d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 11820e32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 11820e32 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 11820e32 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4265 (Fri Sep 28 01:44:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4266 (Fri Sep 28 01:44:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18df8829", "local_conf_timestamp": 4266, "host-ts": 4265}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4261 (Fri Sep 28 01:44:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4260 (Fri Sep 28 01:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "53049f61", "local_conf_timestamp": 4260, "host-ts": 4261}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/352/ Build Number: 352 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4174 (Fri Sep 28 09:43:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4174 (Fri Sep 28 09:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Sep 28 09:43:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4181 (Fri Sep 28 09:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fd299584 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command fd299584 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fe44418a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Fri Sep 28 09:43:38 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4185 (Fri Sep 28 09:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b51bae61", "local_conf_timestamp": 4185, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 05507b9c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Fri Sep 28 09:43:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4195 (Fri Sep 28 09:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c5979ca", "local_conf_timestamp": 4195, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 08641ae6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Fri Sep 28 09:43:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4201 (Fri Sep 28 09:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0bba372a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0ed6b14a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Fri Sep 28 09:44:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Fri Sep 28 09:44:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 11d1e234 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 15a747aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 18a081ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false} lago.ssh: DEBUG: start task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1ba6f34e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false} lago.ssh: DEBUG: start task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1f62be8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false} lago.ssh: DEBUG: start task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 22e5be24 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false} lago.ssh: DEBUG: start task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 25f6d062 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false} lago.ssh: DEBUG: start task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 270d7942 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false} lago.ssh: DEBUG: start task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2e308b56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Fri Sep 28 09:45:01 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4268 (Fri Sep 28 09:45:01 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "d6243e92", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4261 (Fri Sep 28 09:44:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4261 (Fri Sep 28 09:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2403aff7", "local_conf_timestamp": 4261, "host-ts": 4261}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4174 (Fri Sep 28 09:43:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4174 (Fri Sep 28 09:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Sep 28 09:43:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4181 (Fri Sep 28 09:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fd299584 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command fd299584 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fe44418a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Fri Sep 28 09:43:38 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4185 (Fri Sep 28 09:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b51bae61", "local_conf_timestamp": 4185, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 05507b9c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Fri Sep 28 09:43:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4195 (Fri Sep 28 09:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c5979ca", "local_conf_timestamp": 4195, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 08641ae6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Fri Sep 28 09:43:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4201 (Fri Sep 28 09:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0bba372a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0ed6b14a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Fri Sep 28 09:44:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Fri Sep 28 09:44:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 11d1e234 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 15a747aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 18a081ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1ba6f34e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1f62be8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 22e5be24 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 25f6d062 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 270d7942 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2e308b56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Fri Sep 28 09:45:01 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4268 (Fri Sep 28 09:45:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "d6243e92", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4261 (Fri Sep 28 09:44:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4261 (Fri Sep 28 09:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2403aff7", "local_conf_timestamp": 4261, "host-ts": 4261}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/353/ Build Number: 353 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 64ce80d0 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Fri Sep 28 17:46:04 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4206 (Fri Sep 28 17:46:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bf68f918", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Fri Sep 28 17:46:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4207 (Fri Sep 28 17:46:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f933974e", "local_conf_timestamp": 4207, "host-ts": 4207}, "global_maintenance": false} lago.ssh: DEBUG: start task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 66124940 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 66124940 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 67382574 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Fri Sep 28 17:46:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "85a6ce38", "local_conf_timestamp": 4217, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false} lago.ssh: DEBUG: start task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6ea8843e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Sep 28 17:46:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4226 (Fri Sep 28 17:46:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e166f82", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false} lago.ssh: DEBUG: start task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71efd110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false} lago.ssh: DEBUG: start task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 756a0e14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false} lago.ssh: DEBUG: start task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 78530158 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false} lago.ssh: DEBUG: start task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7b5a7214 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false} lago.ssh: DEBUG: start task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7e568156 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false} lago.ssh: DEBUG: start task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 815cfd30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false} lago.ssh: DEBUG: start task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 84889992 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8791ddf6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8db8ea76 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9092e81e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false} lago.ssh: DEBUG: start task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 64ce80d0 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Fri Sep 28 17:46:04 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4206 (Fri Sep 28 17:46:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bf68f918", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Fri Sep 28 17:46:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4207 (Fri Sep 28 17:46:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f933974e", "local_conf_timestamp": 4207, "host-ts": 4207}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 66124940 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 66124940 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 67382574 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Fri Sep 28 17:46:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "85a6ce38", "local_conf_timestamp": 4217, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6ea8843e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Sep 28 17:46:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4226 (Fri Sep 28 17:46:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e166f82", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71efd110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 756a0e14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 78530158 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7b5a7214 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7e568156 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 815cfd30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 84889992 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8791ddf6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8db8ea76 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9092e81e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/354/ Build Number: 354 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:ede22c57-e733-4b6a-9f2f-c5c349fb45ff:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:ede22c57-e733-4b6a-9f2f-c5c349fb45ff:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 5140d818 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5140d818 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 5140d818 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4167 (Sat Sep 29 01:45:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4167 (Sat Sep 29 01:45:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3bf41c38", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4170 (Sat Sep 29 01:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4170 (Sat Sep 29 01:45:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "13bdc3d5", "local_conf_timestamp": 4170, "host-ts": 4170}, "global_maintenance": false} lago.ssh: DEBUG: start task:a95a23c2-8008-4db7-9f49-357c8ea2049a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a95a23c2-8008-4db7-9f49-357c8ea2049a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 52490e7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 52490e7e on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:319ca133-dd93-45eb-a8a7-b72c410a1889:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:319ca133-dd93-45eb-a8a7-b72c410a1889:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 53609cd2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 53609cd2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 53609cd2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Sat Sep 29 01:45:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4177 (Sat Sep 29 01:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7935c07", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Sat Sep 29 01:45:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4180 (Sat Sep 29 01:45:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2c5ac6e6", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false} lago.ssh: DEBUG: start task:ff2643c9-a34f-4b4d-ba4b-bf2adadffc51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ff2643c9-a34f-4b4d-ba4b-bf2adadffc51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5b02e8e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5b02e8e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5b02e8e6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4187 (Sat Sep 29 01:45:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4187 (Sat Sep 29 01:45:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9d4dfe8", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Sat Sep 29 01:45:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4180 (Sat Sep 29 01:45:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2c5ac6e6", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in _wait_for_engine_migration testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 246, in assert_equals_within time.sleep(3) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:ede22c57-e733-4b6a-9f2f-c5c349fb45ff:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:ede22c57-e733-4b6a-9f2f-c5c349fb45ff:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 5140d818 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5140d818 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 5140d818 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4167 (Sat Sep 29 01:45:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4167 (Sat Sep 29 01:45:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3bf41c38", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4170 (Sat Sep 29 01:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4170 (Sat Sep 29 01:45:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "13bdc3d5", "local_conf_timestamp": 4170, "host-ts": 4170}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a95a23c2-8008-4db7-9f49-357c8ea2049a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a95a23c2-8008-4db7-9f49-357c8ea2049a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 52490e7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 52490e7e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:319ca133-dd93-45eb-a8a7-b72c410a1889:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:319ca133-dd93-45eb-a8a7-b72c410a1889:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 53609cd2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 53609cd2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 53609cd2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Sat Sep 29 01:45:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4177 (Sat Sep 29 01:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7935c07", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Sat Sep 29 01:45:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4180 (Sat Sep 29 01:45:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2c5ac6e6", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ff2643c9-a34f-4b4d-ba4b-bf2adadffc51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ff2643c9-a34f-4b4d-ba4b-bf2adadffc51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5b02e8e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5b02e8e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5b02e8e6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4187 (Sat Sep 29 01:45:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4187 (Sat Sep 29 01:45:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9d4dfe8", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Sat Sep 29 01:45:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4180 (Sat Sep 29 01:45:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2c5ac6e6", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/355/ Build Number: 355 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:9b766c4d-71c4-452c-85a0-46f861d13f3d:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:9b766c4d-71c4-452c-85a0-46f861d13f3d:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 9ed7825a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9ed7825a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 9ed7825a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4158 (Sat Sep 29 09:46:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4158 (Sat Sep 29 09:46:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ae1794c7", "local_conf_timestamp": 4158, "host-ts": 4158}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Sat Sep 29 09:46:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4163 (Sat Sep 29 09:46:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "803f2b0b", "local_conf_timestamp": 4163, "host-ts": 4162}, "global_maintenance": false} lago.ssh: DEBUG: start task:003ea7ab-12f9-4749-9377-7a7ef8233b7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:003ea7ab-12f9-4749-9377-7a7ef8233b7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9fe0406a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 9fe0406a on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:6be74bfd-40a6-4f88-a225-dd790e2c27c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6be74bfd-40a6-4f88-a225-dd790e2c27c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a0f68ef0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a0f68ef0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a0f68ef0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4168 (Sat Sep 29 09:47:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4169 (Sat Sep 29 09:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "66c86251", "local_conf_timestamp": 4169, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Sat Sep 29 09:46:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4163 (Sat Sep 29 09:46:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "803f2b0b", "local_conf_timestamp": 4163, "host-ts": 4162}, "global_maintenance": false} lago.ssh: DEBUG: start task:22ea1684-7f3b-49db-b7ac-de50c24cf9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:22ea1684-7f3b-49db-b7ac-de50c24cf9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a84221b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a84221b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a84221b0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Sat Sep 29 09:47:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4178 (Sat Sep 29 09:47:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "eec745d4", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4173 (Sat Sep 29 09:47:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4173 (Sat Sep 29 09:47:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false} lago.ssh: DEBUG: start task:a8b9ea0c-9992-45c1-8dd0-42d0247ad34b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a8b9ea0c-9992-45c1-8dd0-42d0247ad34b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ac5ca22a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ac5ca22a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ac5ca22a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Sat Sep 29 09:47:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Sat Sep 29 09:47:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4183 (Sat Sep 29 09:47:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4183 (Sat Sep 29 09:47:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false} lago.ssh: DEBUG: start task:5dde4be8-3528-4d2c-9997-05892131b78c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5dde4be8-3528-4d2c-9997-05892131b78c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running af4ea514 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command af4ea514 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command af4ea514 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Sat Sep 29 09:47:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Sat Sep 29 09:47:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4183 (Sat Sep 29 09:47:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4183 (Sat Sep 29 09:47:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false} lago.ssh: DEBUG: start task:ff2f5a31-44cc-4a35-b7a6-2eb38e0acab3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ff2f5a31-44cc-4a35-b7a6-2eb38e0acab3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b24578e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b24578e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b24578e2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Sat Sep 29 09:47:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Sat Sep 29 09:47:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4193 (Sat Sep 29 09:47:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4193 (Sat Sep 29 09:47:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "223b3bb0", "local_conf_timestamp": 4193, "host-ts": 4193}, "global_maintenance": false} lago.ssh: DEBUG: start task:702a6cba-d174-4a35-acff-51a056c69c03:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:702a6cba-d174-4a35-acff-51a056c69c03:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b53e1504 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b53e1504 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b53e1504 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Sat Sep 29 09:47:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Sat Sep 29 09:47:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4193 (Sat Sep 29 09:47:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4193 (Sat Sep 29 09:47:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "223b3bb0", "local_conf_timestamp": 4193, "host-ts": 4193}, "global_maintenance": false} lago.ssh: DEBUG: start task:8bf16431-f10e-487f-88b4-821c1e3a97df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8bf16431-f10e-487f-88b4-821c1e3a97df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b828782c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b828782c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b828782c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Sat Sep 29 09:47:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4208 (Sat Sep 29 09:47:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Sat Sep 29 09:47:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4204 (Sat Sep 29 09:47:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0a9d7fae", "local_conf_timestamp": 4204, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:3d7d9417-f3b0-4763-90f0-e6906bcc757b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3d7d9417-f3b0-4763-90f0-e6906bcc757b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bb3edcf4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bb3edcf4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bb3edcf4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Sat Sep 29 09:47:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4208 (Sat Sep 29 09:47:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Sat Sep 29 09:47:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4204 (Sat Sep 29 09:47:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0a9d7fae", "local_conf_timestamp": 4204, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:ed6520f7-6d46-4a53-ae34-1a818a045804:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ed6520f7-6d46-4a53-ae34-1a818a045804:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running be1a172c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command be1a172c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command be1a172c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Sat Sep 29 09:47:54 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Sat Sep 29 09:47:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Sat Sep 29 09:47:49 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4213 (Sat Sep 29 09:47:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:570dd3c0-26c1-485b-9c06-de106860636e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:570dd3c0-26c1-485b-9c06-de106860636e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c115ed7a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c115ed7a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c115ed7a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Sat Sep 29 09:47:54 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Sat Sep 29 09:47:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Sat Sep 29 09:47:49 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4213 (Sat Sep 29 09:47:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:45c168c1-5908-44a2-bf55-97a49e40b93f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:45c168c1-5908-44a2-bf55-97a49e40b93f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c4aa904e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c4aa904e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c4aa904e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sat Sep 29 09:48:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sat Sep 29 09:48:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Sat Sep 29 09:47:59 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4223 (Sat Sep 29 09:47:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:c30c38cb-7f45-488e-8e64-3b746718a101:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c30c38cb-7f45-488e-8e64-3b746718a101:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c7cd5946 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c7cd5946 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c7cd5946 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sat Sep 29 09:48:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sat Sep 29 09:48:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Sat Sep 29 09:47:59 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4223 (Sat Sep 29 09:47:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:68c70086-ff3e-497a-8c6f-881b8c688f9b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:68c70086-ff3e-497a-8c6f-881b8c688f9b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running caef7e10 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command caef7e10 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command caef7e10 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Sat Sep 29 09:48:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4239 (Sat Sep 29 09:48:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c12ce8d0", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Sat Sep 29 09:48:09 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4233 (Sat Sep 29 09:48:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:fd511798-81f6-41e0-ab71-dc6627539b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fd511798-81f6-41e0-ab71-dc6627539b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cdced306 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cdced306 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cdced306 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Sat Sep 29 09:48:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4239 (Sat Sep 29 09:48:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c12ce8d0", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Sat Sep 29 09:48:09 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4233 (Sat Sep 29 09:48:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:5e7b3406-1b12-464c-95d4-d79a297cff0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5e7b3406-1b12-464c-95d4-d79a297cff0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d0adc9ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d0adc9ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d0adc9ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Sat Sep 29 09:48:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4248 (Sat Sep 29 09:48:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3b9bee72", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Sat Sep 29 09:48:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4244 (Sat Sep 29 09:48:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e55139d", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false} lago.ssh: DEBUG: start task:3feec685-cad9-472f-9691-f3952fe40972:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3feec685-cad9-472f-9691-f3952fe40972:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d1e569d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d1e569d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d1e569d2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Sat Sep 29 09:48:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4248 (Sat Sep 29 09:48:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3b9bee72", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Sat Sep 29 09:48:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4244 (Sat Sep 29 09:48:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e55139d", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:9b766c4d-71c4-452c-85a0-46f861d13f3d:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:9b766c4d-71c4-452c-85a0-46f861d13f3d:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 9ed7825a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9ed7825a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 9ed7825a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4158 (Sat Sep 29 09:46:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4158 (Sat Sep 29 09:46:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ae1794c7", "local_conf_timestamp": 4158, "host-ts": 4158}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Sat Sep 29 09:46:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4163 (Sat Sep 29 09:46:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "803f2b0b", "local_conf_timestamp": 4163, "host-ts": 4162}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:003ea7ab-12f9-4749-9377-7a7ef8233b7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:003ea7ab-12f9-4749-9377-7a7ef8233b7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9fe0406a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 9fe0406a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:6be74bfd-40a6-4f88-a225-dd790e2c27c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6be74bfd-40a6-4f88-a225-dd790e2c27c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a0f68ef0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a0f68ef0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a0f68ef0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4168 (Sat Sep 29 09:47:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4169 (Sat Sep 29 09:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "66c86251", "local_conf_timestamp": 4169, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Sat Sep 29 09:46:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4163 (Sat Sep 29 09:46:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "803f2b0b", "local_conf_timestamp": 4163, "host-ts": 4162}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:22ea1684-7f3b-49db-b7ac-de50c24cf9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:22ea1684-7f3b-49db-b7ac-de50c24cf9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a84221b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a84221b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a84221b0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Sat Sep 29 09:47:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4178 (Sat Sep 29 09:47:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "eec745d4", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4173 (Sat Sep 29 09:47:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4173 (Sat Sep 29 09:47:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a8b9ea0c-9992-45c1-8dd0-42d0247ad34b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a8b9ea0c-9992-45c1-8dd0-42d0247ad34b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ac5ca22a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ac5ca22a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ac5ca22a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Sat Sep 29 09:47:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Sat Sep 29 09:47:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4183 (Sat Sep 29 09:47:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4183 (Sat Sep 29 09:47:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5dde4be8-3528-4d2c-9997-05892131b78c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5dde4be8-3528-4d2c-9997-05892131b78c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running af4ea514 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command af4ea514 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command af4ea514 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Sat Sep 29 09:47:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Sat Sep 29 09:47:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4183 (Sat Sep 29 09:47:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4183 (Sat Sep 29 09:47:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ff2f5a31-44cc-4a35-b7a6-2eb38e0acab3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ff2f5a31-44cc-4a35-b7a6-2eb38e0acab3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b24578e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b24578e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b24578e2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Sat Sep 29 09:47:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Sat Sep 29 09:47:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4193 (Sat Sep 29 09:47:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4193 (Sat Sep 29 09:47:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "223b3bb0", "local_conf_timestamp": 4193, "host-ts": 4193}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:702a6cba-d174-4a35-acff-51a056c69c03:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:702a6cba-d174-4a35-acff-51a056c69c03:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b53e1504 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b53e1504 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b53e1504 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Sat Sep 29 09:47:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Sat Sep 29 09:47:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4193 (Sat Sep 29 09:47:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4193 (Sat Sep 29 09:47:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "223b3bb0", "local_conf_timestamp": 4193, "host-ts": 4193}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8bf16431-f10e-487f-88b4-821c1e3a97df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8bf16431-f10e-487f-88b4-821c1e3a97df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b828782c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b828782c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b828782c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Sat Sep 29 09:47:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4208 (Sat Sep 29 09:47:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Sat Sep 29 09:47:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4204 (Sat Sep 29 09:47:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0a9d7fae", "local_conf_timestamp": 4204, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3d7d9417-f3b0-4763-90f0-e6906bcc757b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3d7d9417-f3b0-4763-90f0-e6906bcc757b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bb3edcf4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bb3edcf4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bb3edcf4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Sat Sep 29 09:47:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4208 (Sat Sep 29 09:47:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Sat Sep 29 09:47:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4204 (Sat Sep 29 09:47:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0a9d7fae", "local_conf_timestamp": 4204, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ed6520f7-6d46-4a53-ae34-1a818a045804:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ed6520f7-6d46-4a53-ae34-1a818a045804:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running be1a172c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command be1a172c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command be1a172c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Sat Sep 29 09:47:54 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Sat Sep 29 09:47:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Sat Sep 29 09:47:49 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4213 (Sat Sep 29 09:47:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:570dd3c0-26c1-485b-9c06-de106860636e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:570dd3c0-26c1-485b-9c06-de106860636e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c115ed7a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c115ed7a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c115ed7a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Sat Sep 29 09:47:54 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Sat Sep 29 09:47:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Sat Sep 29 09:47:49 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4213 (Sat Sep 29 09:47:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:45c168c1-5908-44a2-bf55-97a49e40b93f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:45c168c1-5908-44a2-bf55-97a49e40b93f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c4aa904e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c4aa904e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c4aa904e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sat Sep 29 09:48:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sat Sep 29 09:48:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Sat Sep 29 09:47:59 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4223 (Sat Sep 29 09:47:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c30c38cb-7f45-488e-8e64-3b746718a101:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c30c38cb-7f45-488e-8e64-3b746718a101:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c7cd5946 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c7cd5946 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c7cd5946 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sat Sep 29 09:48:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sat Sep 29 09:48:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Sat Sep 29 09:47:59 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4223 (Sat Sep 29 09:47:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:68c70086-ff3e-497a-8c6f-881b8c688f9b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:68c70086-ff3e-497a-8c6f-881b8c688f9b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running caef7e10 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command caef7e10 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command caef7e10 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Sat Sep 29 09:48:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4239 (Sat Sep 29 09:48:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c12ce8d0", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Sat Sep 29 09:48:09 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4233 (Sat Sep 29 09:48:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fd511798-81f6-41e0-ab71-dc6627539b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fd511798-81f6-41e0-ab71-dc6627539b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cdced306 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cdced306 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cdced306 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Sat Sep 29 09:48:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4239 (Sat Sep 29 09:48:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c12ce8d0", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Sat Sep 29 09:48:09 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4233 (Sat Sep 29 09:48:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5e7b3406-1b12-464c-95d4-d79a297cff0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5e7b3406-1b12-464c-95d4-d79a297cff0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d0adc9ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d0adc9ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d0adc9ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Sat Sep 29 09:48:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4248 (Sat Sep 29 09:48:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3b9bee72", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Sat Sep 29 09:48:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4244 (Sat Sep 29 09:48:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e55139d", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3feec685-cad9-472f-9691-f3952fe40972:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3feec685-cad9-472f-9691-f3952fe40972:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d1e569d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d1e569d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d1e569d2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Sat Sep 29 09:48:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4248 (Sat Sep 29 09:48:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3b9bee72", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Sat Sep 29 09:48:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4244 (Sat Sep 29 09:48:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e55139d", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/356/ Build Number: 356 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/357/ Build Number: 357 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 0350105a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4159 (Sun Sep 30 01:41:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4159 (Sun Sep 30 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false} lago.ssh: DEBUG: start task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 04612d44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 04612d44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 05749c84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false} lago.ssh: DEBUG: start task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0c910dcc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sun Sep 30 01:42:02 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4175 (Sun Sep 30 01:42:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d16ed36", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false} lago.ssh: DEBUG: start task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0f9a3bce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false} lago.ssh: DEBUG: start task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 127d027c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false} lago.ssh: DEBUG: start task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 15ef7d7c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Sep 30 01:42:22 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4196 (Sun Sep 30 01:42:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Sun Sep 30 01:42:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4189 (Sun Sep 30 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f350724e", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false} lago.ssh: DEBUG: start task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 18f6d48e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1f13ce1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 223f3bee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 25806706 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false} lago.ssh: DEBUG: start task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 28a3f2cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false} lago.ssh: DEBUG: start task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 33ce4562 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Sun Sep 30 01:43:12 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4245 (Sun Sep 30 01:43:12 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "df015cca", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Sun Sep 30 01:43:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4240 (Sun Sep 30 01:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f6f1d9e6", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 0350105a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4159 (Sun Sep 30 01:41:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4159 (Sun Sep 30 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 04612d44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 04612d44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 05749c84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0c910dcc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sun Sep 30 01:42:02 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4175 (Sun Sep 30 01:42:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d16ed36", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0f9a3bce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 127d027c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 15ef7d7c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Sep 30 01:42:22 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4196 (Sun Sep 30 01:42:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Sun Sep 30 01:42:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4189 (Sun Sep 30 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f350724e", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 18f6d48e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1f13ce1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 223f3bee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 25806706 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 28a3f2cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 33ce4562 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Sun Sep 30 01:43:12 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4245 (Sun Sep 30 01:43:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "df015cca", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Sun Sep 30 01:43:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4240 (Sun Sep 30 01:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f6f1d9e6", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/358/ Build Number: 358 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/359/ Build Number: 359 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:351db145-144d-4f08-a64d-872742c93417:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:351db145-144d-4f08-a64d-872742c93417:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running b830699a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b830699a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command b830699a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Sun Sep 30 17:38:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4166 (Sun Sep 30 17:38:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b17ab9f", "local_conf_timestamp": 4166, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4173 (Sun Sep 30 17:38:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4173 (Sun Sep 30 17:38:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false} lago.ssh: DEBUG: start task:72deca2e-82f8-4687-8a75-3123128c7bc6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:72deca2e-82f8-4687-8a75-3123128c7bc6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b92f2534 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command b92f2534 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:383ec35a-b4a9-4a8c-a1ed-a7b467f30925:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:383ec35a-b4a9-4a8c-a1ed-a7b467f30925:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ba3276b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ba3276b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ba3276b6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sun Sep 30 17:38:57 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4175 (Sun Sep 30 17:38:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e79ba1a", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4173 (Sun Sep 30 17:38:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4173 (Sun Sep 30 17:38:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false} lago.ssh: DEBUG: start task:8b4115a0-271c-47ae-bf42-5eed877e3d4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8b4115a0-271c-47ae-bf42-5eed877e3d4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c214b3b2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c214b3b2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c214b3b2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Sep 30 17:39:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4196 (Sun Sep 30 17:39:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4193 (Sun Sep 30 17:39:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4194 (Sun Sep 30 17:39:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cb629b8b", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false} lago.ssh: DEBUG: start task:b61e9ba0-a5d2-4bf2-a7aa-9d769f503f1d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b61e9ba0-a5d2-4bf2-a7aa-9d769f503f1d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c526ad58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c526ad58 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c526ad58 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Sep 30 17:39:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4196 (Sun Sep 30 17:39:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4193 (Sun Sep 30 17:39:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4194 (Sun Sep 30 17:39:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cb629b8b", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false} lago.ssh: DEBUG: start task:86087f96-02d7-4271-85fa-5c613503cd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:86087f96-02d7-4271-85fa-5c613503cd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c832f024 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c832f024 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c832f024 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 30 17:39:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Sun Sep 30 17:39:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4204 (Sun Sep 30 17:39:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4204 (Sun Sep 30 17:39:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f314951d", "local_conf_timestamp": 4204, "host-ts": 4204}, "global_maintenance": false} lago.ssh: DEBUG: start task:e4d93de1-b3e0-478d-b45a-a271253b8681:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e4d93de1-b3e0-478d-b45a-a271253b8681:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb1928a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cb1928a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cb1928a8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 30 17:39:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Sun Sep 30 17:39:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4204 (Sun Sep 30 17:39:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4204 (Sun Sep 30 17:39:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f314951d", "local_conf_timestamp": 4204, "host-ts": 4204}, "global_maintenance": false} lago.ssh: DEBUG: start task:85226c0f-3d5f-480f-a9ac-53c166ec8d21:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:85226c0f-3d5f-480f-a9ac-53c166ec8d21:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ce14d480 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ce14d480 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ce14d480 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 30 17:39:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Sun Sep 30 17:39:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4214 (Sun Sep 30 17:39:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4214 (Sun Sep 30 17:39:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7526a18", "local_conf_timestamp": 4214, "host-ts": 4214}, "global_maintenance": false} lago.ssh: DEBUG: start task:b30d1798-cc2c-41c5-bd17-d93e0ad8e748:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b30d1798-cc2c-41c5-bd17-d93e0ad8e748:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d0efcb88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d0efcb88 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d0efcb88 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 30 17:39:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Sun Sep 30 17:39:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4214 (Sun Sep 30 17:39:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4214 (Sun Sep 30 17:39:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7526a18", "local_conf_timestamp": 4214, "host-ts": 4214}, "global_maintenance": false} lago.ssh: DEBUG: start task:40baa0e7-6867-4d5c-b6da-063f2eb3ea58:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:40baa0e7-6867-4d5c-b6da-063f2eb3ea58:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d3ef38d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d3ef38d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d3ef38d2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 30 17:39:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 30 17:39:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Sun Sep 30 17:39:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4224 (Sun Sep 30 17:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48bdb399", "local_conf_timestamp": 4224, "host-ts": 4224}, "global_maintenance": false} lago.ssh: DEBUG: start task:91fc8789-2a00-403e-bba0-e0033a287760:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:91fc8789-2a00-403e-bba0-e0033a287760:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d6d7d824 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d6d7d824 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d6d7d824 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 30 17:39:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 30 17:39:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Sun Sep 30 17:39:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4224 (Sun Sep 30 17:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48bdb399", "local_conf_timestamp": 4224, "host-ts": 4224}, "global_maintenance": false} lago.ssh: DEBUG: start task:537859e3-1c5e-402e-a70c-c638e7098ea7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:537859e3-1c5e-402e-a70c-c638e7098ea7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d9ee642e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d9ee642e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d9ee642e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 30 17:39:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Sun Sep 30 17:39:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Sun Sep 30 17:39:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4235 (Sun Sep 30 17:39:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3bfc7c0a", "local_conf_timestamp": 4235, "host-ts": 4234}, "global_maintenance": false} lago.ssh: DEBUG: start task:0b1011b0-54da-4352-a438-5aaeeb4791c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0b1011b0-54da-4352-a438-5aaeeb4791c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dd2386ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dd2386ec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dd2386ec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 30 17:39:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Sun Sep 30 17:39:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Sun Sep 30 17:39:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4235 (Sun Sep 30 17:39:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3bfc7c0a", "local_conf_timestamp": 4235, "host-ts": 4234}, "global_maintenance": false} lago.ssh: DEBUG: start task:a65f6863-f1b3-4aa7-b94d-3ff202b0fb01:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a65f6863-f1b3-4aa7-b94d-3ff202b0fb01:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e044d510 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e044d510 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e044d510 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Sun Sep 30 17:40:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4245 (Sun Sep 30 17:40:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e8d49504", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Sun Sep 30 17:40:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4244 (Sun Sep 30 17:40:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "512bb187", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false} lago.ssh: DEBUG: start task:f265d09e-b398-4040-9297-8e5500fe9cc0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f265d09e-b398-4040-9297-8e5500fe9cc0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e3270a82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e3270a82 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e3270a82 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Sun Sep 30 17:40:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4245 (Sun Sep 30 17:40:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e8d49504", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Sun Sep 30 17:40:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4244 (Sun Sep 30 17:40:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "512bb187", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false} lago.ssh: DEBUG: start task:d2849bb1-3b2e-42e8-bb6d-02975bb3878e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d2849bb1-3b2e-42e8-bb6d-02975bb3878e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e60bda52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e60bda52 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e60bda52 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Sun Sep 30 17:40:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4255 (Sun Sep 30 17:40:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775aa885", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4254 (Sun Sep 30 17:40:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4254 (Sun Sep 30 17:40:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "823253a2", "local_conf_timestamp": 4254, "host-ts": 4254}, "global_maintenance": false} lago.ssh: DEBUG: start task:ded09d38-29ed-404b-8d53-ee5befacdb9c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ded09d38-29ed-404b-8d53-ee5befacdb9c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e7261466 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e7261466 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e7261466 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Sun Sep 30 17:40:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4255 (Sun Sep 30 17:40:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775aa885", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4254 (Sun Sep 30 17:40:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4254 (Sun Sep 30 17:40:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "823253a2", "local_conf_timestamp": 4254, "host-ts": 4254}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:f348f612-9829-4c8a-9298-02b3a5464e20:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f348f612-9829-4c8a-9298-02b3a5464e20:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9b127ab4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none lago.ssh: DEBUG: Command 9b127ab4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for Engine Maintenance to reset... lago.ssh: DEBUG: start task:85671d69-7ed2-4641-a0e1-ebdca009df24:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:85671d69-7ed2-4641-a0e1-ebdca009df24:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9d52ba82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9d52ba82 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9d52ba82 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4556 (Sun Sep 30 17:45:18 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4556 (Sun Sep 30 17:45:18 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bd852b75", "local_conf_timestamp": 4556, "host-ts": 4556}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4555 (Sun Sep 30 17:45:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4555 (Sun Sep 30 17:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "84ba587d", "local_conf_timestamp": 4555, "host-ts": 4555}, "global_maintenance": false} lago.ssh: DEBUG: start task:fbb429da-32fe-4dd1-a982-33e04700c5db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fbb429da-32fe-4dd1-a982-33e04700c5db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 130, in local_maintenance _wait_for_engine_maintenance(host, he_index, False) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 69, in _wait_for_engine_maintenance testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 69, in <lambda> testlib.assert_true_within_long(lambda: _get_he_status(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 92, in _get_he_status ret = host.ssh(["hosted-engine", "--vm-status", "--json"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 49, in ssh channel = transport.open_session() File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 712, in open_session timeout=timeout) File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 828, in open_channel event.wait(0.1) File "/usr/lib64/python2.7/threading.py", line 622, in wait self.__cond.wait(timeout, balancing) File "/usr/lib64/python2.7/threading.py", line 362, in wait _sleep(delay) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:351db145-144d-4f08-a64d-872742c93417:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:351db145-144d-4f08-a64d-872742c93417:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running b830699a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b830699a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command b830699a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Sun Sep 30 17:38:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4166 (Sun Sep 30 17:38:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b17ab9f", "local_conf_timestamp": 4166, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4173 (Sun Sep 30 17:38:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4173 (Sun Sep 30 17:38:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:72deca2e-82f8-4687-8a75-3123128c7bc6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:72deca2e-82f8-4687-8a75-3123128c7bc6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b92f2534 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command b92f2534 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:383ec35a-b4a9-4a8c-a1ed-a7b467f30925:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:383ec35a-b4a9-4a8c-a1ed-a7b467f30925:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ba3276b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ba3276b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ba3276b6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sun Sep 30 17:38:57 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4175 (Sun Sep 30 17:38:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e79ba1a", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4173 (Sun Sep 30 17:38:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4173 (Sun Sep 30 17:38:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8b4115a0-271c-47ae-bf42-5eed877e3d4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8b4115a0-271c-47ae-bf42-5eed877e3d4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c214b3b2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c214b3b2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c214b3b2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Sep 30 17:39:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4196 (Sun Sep 30 17:39:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4193 (Sun Sep 30 17:39:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4194 (Sun Sep 30 17:39:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cb629b8b", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b61e9ba0-a5d2-4bf2-a7aa-9d769f503f1d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b61e9ba0-a5d2-4bf2-a7aa-9d769f503f1d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c526ad58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c526ad58 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c526ad58 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Sep 30 17:39:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4196 (Sun Sep 30 17:39:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4193 (Sun Sep 30 17:39:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4194 (Sun Sep 30 17:39:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cb629b8b", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:86087f96-02d7-4271-85fa-5c613503cd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:86087f96-02d7-4271-85fa-5c613503cd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c832f024 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c832f024 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c832f024 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 30 17:39:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Sun Sep 30 17:39:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4204 (Sun Sep 30 17:39:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4204 (Sun Sep 30 17:39:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f314951d", "local_conf_timestamp": 4204, "host-ts": 4204}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e4d93de1-b3e0-478d-b45a-a271253b8681:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e4d93de1-b3e0-478d-b45a-a271253b8681:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb1928a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cb1928a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cb1928a8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 30 17:39:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Sun Sep 30 17:39:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4204 (Sun Sep 30 17:39:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4204 (Sun Sep 30 17:39:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f314951d", "local_conf_timestamp": 4204, "host-ts": 4204}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:85226c0f-3d5f-480f-a9ac-53c166ec8d21:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:85226c0f-3d5f-480f-a9ac-53c166ec8d21:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ce14d480 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ce14d480 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ce14d480 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 30 17:39:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Sun Sep 30 17:39:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4214 (Sun Sep 30 17:39:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4214 (Sun Sep 30 17:39:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7526a18", "local_conf_timestamp": 4214, "host-ts": 4214}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b30d1798-cc2c-41c5-bd17-d93e0ad8e748:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b30d1798-cc2c-41c5-bd17-d93e0ad8e748:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d0efcb88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d0efcb88 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d0efcb88 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 30 17:39:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Sun Sep 30 17:39:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4214 (Sun Sep 30 17:39:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4214 (Sun Sep 30 17:39:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7526a18", "local_conf_timestamp": 4214, "host-ts": 4214}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:40baa0e7-6867-4d5c-b6da-063f2eb3ea58:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:40baa0e7-6867-4d5c-b6da-063f2eb3ea58:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d3ef38d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d3ef38d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d3ef38d2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 30 17:39:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 30 17:39:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Sun Sep 30 17:39:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4224 (Sun Sep 30 17:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48bdb399", "local_conf_timestamp": 4224, "host-ts": 4224}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:91fc8789-2a00-403e-bba0-e0033a287760:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:91fc8789-2a00-403e-bba0-e0033a287760:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d6d7d824 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d6d7d824 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d6d7d824 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 30 17:39:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 30 17:39:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Sun Sep 30 17:39:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4224 (Sun Sep 30 17:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48bdb399", "local_conf_timestamp": 4224, "host-ts": 4224}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:537859e3-1c5e-402e-a70c-c638e7098ea7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:537859e3-1c5e-402e-a70c-c638e7098ea7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d9ee642e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d9ee642e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d9ee642e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 30 17:39:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Sun Sep 30 17:39:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Sun Sep 30 17:39:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4235 (Sun Sep 30 17:39:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3bfc7c0a", "local_conf_timestamp": 4235, "host-ts": 4234}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0b1011b0-54da-4352-a438-5aaeeb4791c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0b1011b0-54da-4352-a438-5aaeeb4791c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dd2386ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dd2386ec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dd2386ec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 30 17:39:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Sun Sep 30 17:39:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Sun Sep 30 17:39:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4235 (Sun Sep 30 17:39:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3bfc7c0a", "local_conf_timestamp": 4235, "host-ts": 4234}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a65f6863-f1b3-4aa7-b94d-3ff202b0fb01:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a65f6863-f1b3-4aa7-b94d-3ff202b0fb01:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e044d510 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e044d510 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e044d510 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Sun Sep 30 17:40:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4245 (Sun Sep 30 17:40:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e8d49504", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Sun Sep 30 17:40:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4244 (Sun Sep 30 17:40:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "512bb187", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f265d09e-b398-4040-9297-8e5500fe9cc0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f265d09e-b398-4040-9297-8e5500fe9cc0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e3270a82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e3270a82 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e3270a82 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Sun Sep 30 17:40:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4245 (Sun Sep 30 17:40:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e8d49504", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Sun Sep 30 17:40:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4244 (Sun Sep 30 17:40:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "512bb187", "local_conf_timestamp": 4244, "host-ts": 4244}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d2849bb1-3b2e-42e8-bb6d-02975bb3878e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d2849bb1-3b2e-42e8-bb6d-02975bb3878e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e60bda52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e60bda52 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e60bda52 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Sun Sep 30 17:40:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4255 (Sun Sep 30 17:40:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775aa885", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4254 (Sun Sep 30 17:40:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4254 (Sun Sep 30 17:40:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "823253a2", "local_conf_timestamp": 4254, "host-ts": 4254}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ded09d38-29ed-404b-8d53-ee5befacdb9c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ded09d38-29ed-404b-8d53-ee5befacdb9c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e7261466 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e7261466 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e7261466 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Sun Sep 30 17:40:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4255 (Sun Sep 30 17:40:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775aa885", "local_conf_timestamp": 4255, "host-ts": 4255}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4254 (Sun Sep 30 17:40:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4254 (Sun Sep 30 17:40:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "823253a2", "local_conf_timestamp": 4254, "host-ts": 4254}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:f348f612-9829-4c8a-9298-02b3a5464e20:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f348f612-9829-4c8a-9298-02b3a5464e20:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9b127ab4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none\nlago.ssh: DEBUG: Command 9b127ab4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for Engine Maintenance to reset...\nlago.ssh: DEBUG: start task:85671d69-7ed2-4641-a0e1-ebdca009df24:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:85671d69-7ed2-4641-a0e1-ebdca009df24:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9d52ba82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9d52ba82 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9d52ba82 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4556 (Sun Sep 30 17:45:18 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4556 (Sun Sep 30 17:45:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bd852b75", "local_conf_timestamp": 4556, "host-ts": 4556}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4555 (Sun Sep 30 17:45:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4555 (Sun Sep 30 17:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "84ba587d", "local_conf_timestamp": 4555, "host-ts": 4555}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fbb429da-32fe-4dd1-a982-33e04700c5db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fbb429da-32fe-4dd1-a982-33e04700c5db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/360/ Build Number: 360 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:d1cf76eb-17a1-4d80-8258-2d038105a87a:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:d1cf76eb-17a1-4d80-8258-2d038105a87a:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 990957c2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 990957c2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 990957c2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Mon Oct 1 01:44:44 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4177 (Mon Oct 1 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "21d99cbf", "local_conf_timestamp": 4177, "host-ts": 4176}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Mon Oct 1 01:44:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Mon Oct 1 01:44:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "288a87da", "local_conf_timestamp": 4182, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:b25726ee-9b63-4a7e-975f-58995e9442e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b25726ee-9b63-4a7e-975f-58995e9442e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9a1393f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 9a1393f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:5a8621bd-6790-453d-ba10-38ab40f54d77:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5a8621bd-6790-453d-ba10-38ab40f54d77:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9b27e3ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9b27e3ac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9b27e3ac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4186 (Mon Oct 1 01:44:53 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4186 (Mon Oct 1 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7bf678ea", "local_conf_timestamp": 4186, "host-ts": 4186}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Mon Oct 1 01:44:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Mon Oct 1 01:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:88666d3d-b20a-421e-aed1-c1d731b473fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:88666d3d-b20a-421e-aed1-c1d731b473fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a2749826 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a2749826 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a2749826 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Mon Oct 1 01:45:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4196 (Mon Oct 1 01:45:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "84fe6ff9", "local_conf_timestamp": 4196, "host-ts": 4196}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Mon Oct 1 01:44:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Mon Oct 1 01:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:ea0e70f8-09c2-4023-a50f-736ab5f22b18:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ea0e70f8-09c2-4023-a50f-736ab5f22b18:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a5a3344e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a5a3344e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a5a3344e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Mon Oct 1 01:45:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Mon Oct 1 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b700cb20", "local_conf_timestamp": 4207, "host-ts": 4206}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Mon Oct 1 01:45:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4201 (Mon Oct 1 01:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:d23d868f-9583-4244-85be-030ee6ebc018:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d23d868f-9583-4244-85be-030ee6ebc018:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a8a91cbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a8a91cbc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a8a91cbc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Mon Oct 1 01:45:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Mon Oct 1 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b700cb20", "local_conf_timestamp": 4207, "host-ts": 4206}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Mon Oct 1 01:45:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4201 (Mon Oct 1 01:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:70c7a830-137a-4399-8154-a5645cbc8815:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:70c7a830-137a-4399-8154-a5645cbc8815:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aba91db8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aba91db8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command aba91db8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Mon Oct 1 01:45:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Mon Oct 1 01:45:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e34c574", "local_conf_timestamp": 4216, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Mon Oct 1 01:45:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Mon Oct 1 01:45:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:7080ca90-5c7d-42a3-ad81-32908f811989:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7080ca90-5c7d-42a3-ad81-32908f811989:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aea0ef8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aea0ef8c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command aea0ef8c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Mon Oct 1 01:45:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Mon Oct 1 01:45:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e34c574", "local_conf_timestamp": 4216, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Mon Oct 1 01:45:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Mon Oct 1 01:45:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:d309bf02-93c0-447a-9d0d-cf577fdf06ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d309bf02-93c0-447a-9d0d-cf577fdf06ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b235546c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in _wait_for_engine_migration testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in <lambda> testlib.assert_true_within_long(lambda: _get_he_status(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 92, in _get_he_status ret = host.ssh(["hosted-engine", "--vm-status", "--json"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 69, in ssh 'stderr': None File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 247, in drain_ssh_channel 0.1, File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:d1cf76eb-17a1-4d80-8258-2d038105a87a:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:d1cf76eb-17a1-4d80-8258-2d038105a87a:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 990957c2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 990957c2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 990957c2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Mon Oct 1 01:44:44 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4177 (Mon Oct 1 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "21d99cbf", "local_conf_timestamp": 4177, "host-ts": 4176}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Mon Oct 1 01:44:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Mon Oct 1 01:44:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "288a87da", "local_conf_timestamp": 4182, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b25726ee-9b63-4a7e-975f-58995e9442e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b25726ee-9b63-4a7e-975f-58995e9442e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9a1393f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 9a1393f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:5a8621bd-6790-453d-ba10-38ab40f54d77:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5a8621bd-6790-453d-ba10-38ab40f54d77:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9b27e3ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9b27e3ac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9b27e3ac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4186 (Mon Oct 1 01:44:53 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4186 (Mon Oct 1 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7bf678ea", "local_conf_timestamp": 4186, "host-ts": 4186}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Mon Oct 1 01:44:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Mon Oct 1 01:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:88666d3d-b20a-421e-aed1-c1d731b473fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:88666d3d-b20a-421e-aed1-c1d731b473fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a2749826 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a2749826 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a2749826 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Mon Oct 1 01:45:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4196 (Mon Oct 1 01:45:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "84fe6ff9", "local_conf_timestamp": 4196, "host-ts": 4196}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Mon Oct 1 01:44:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Mon Oct 1 01:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ea0e70f8-09c2-4023-a50f-736ab5f22b18:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ea0e70f8-09c2-4023-a50f-736ab5f22b18:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a5a3344e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a5a3344e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a5a3344e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Mon Oct 1 01:45:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Mon Oct 1 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b700cb20", "local_conf_timestamp": 4207, "host-ts": 4206}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Mon Oct 1 01:45:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4201 (Mon Oct 1 01:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d23d868f-9583-4244-85be-030ee6ebc018:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d23d868f-9583-4244-85be-030ee6ebc018:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a8a91cbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a8a91cbc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a8a91cbc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Mon Oct 1 01:45:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Mon Oct 1 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b700cb20", "local_conf_timestamp": 4207, "host-ts": 4206}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Mon Oct 1 01:45:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4201 (Mon Oct 1 01:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:70c7a830-137a-4399-8154-a5645cbc8815:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:70c7a830-137a-4399-8154-a5645cbc8815:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aba91db8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aba91db8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command aba91db8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Mon Oct 1 01:45:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Mon Oct 1 01:45:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e34c574", "local_conf_timestamp": 4216, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Mon Oct 1 01:45:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Mon Oct 1 01:45:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7080ca90-5c7d-42a3-ad81-32908f811989:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7080ca90-5c7d-42a3-ad81-32908f811989:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aea0ef8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aea0ef8c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command aea0ef8c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Mon Oct 1 01:45:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Mon Oct 1 01:45:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e34c574", "local_conf_timestamp": 4216, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Mon Oct 1 01:45:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Mon Oct 1 01:45:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d309bf02-93c0-447a-9d0d-cf577fdf06ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d309bf02-93c0-447a-9d0d-cf577fdf06ee:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b235546c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/361/ Build Number: 361 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.clear_global_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 76, in clear_global_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/362/ Build Number: 362 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:3becef3a-a3bc-451d-a134-b37387ff9a6c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:3becef3a-a3bc-451d-a134-b37387ff9a6c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running a167d066 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a167d066 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command a167d066 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4148 (Mon Oct 1 17:44:15 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4148 (Mon Oct 1 17:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7270d4f8", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4151 (Mon Oct 1 17:44:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4151 (Mon Oct 1 17:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "86e92d59", "local_conf_timestamp": 4151, "host-ts": 4151}, "global_maintenance": false} lago.ssh: DEBUG: start task:6908cdbc-d88c-4fe6-809f-dec0c42c43dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6908cdbc-d88c-4fe6-809f-dec0c42c43dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a2a75e24 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command a2a75e24 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:a6239f1a-9d61-4224-8d37-45d222f2927f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a6239f1a-9d61-4224-8d37-45d222f2927f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a3d02826 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a3d02826 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a3d02826 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4159 (Mon Oct 1 17:44:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4159 (Mon Oct 1 17:44:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3c3533c5", "local_conf_timestamp": 4159, "host-ts": 4159}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4151 (Mon Oct 1 17:44:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4151 (Mon Oct 1 17:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "86e92d59", "local_conf_timestamp": 4151, "host-ts": 4151}, "global_maintenance": false} lago.ssh: DEBUG: start task:b2d0f62d-f55e-4c2b-a2e2-e1038dd1481f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b2d0f62d-f55e-4c2b-a2e2-e1038dd1481f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ab0e8de4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ab0e8de4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ab0e8de4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Mon Oct 1 17:44:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4169 (Mon Oct 1 17:44:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b15e2e41", "local_conf_timestamp": 4169, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4161 (Mon Oct 1 17:44:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4161 (Mon Oct 1 17:44:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8a222c56", "local_conf_timestamp": 4161, "host-ts": 4161}, "global_maintenance": false} lago.ssh: DEBUG: start task:e1de5fb6-a808-4e10-973e-65371f552b71:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e1de5fb6-a808-4e10-973e-65371f552b71:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ae15c138 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ae15c138 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ae15c138 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Mon Oct 1 17:44:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4169 (Mon Oct 1 17:44:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b15e2e41", "local_conf_timestamp": 4169, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4161 (Mon Oct 1 17:44:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4161 (Mon Oct 1 17:44:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8a222c56", "local_conf_timestamp": 4161, "host-ts": 4161}, "global_maintenance": false} lago.ssh: DEBUG: start task:459dcf92-4a1d-44a8-8c57-37c4a8d94644:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:459dcf92-4a1d-44a8-8c57-37c4a8d94644:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b144dc18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b144dc18 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b144dc18 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Mon Oct 1 17:44:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4180 (Mon Oct 1 17:44:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e129b4e8", "local_conf_timestamp": 4180, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4171 (Mon Oct 1 17:44:38 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4171 (Mon Oct 1 17:44:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f959e6cb", "local_conf_timestamp": 4171, "host-ts": 4171}, "global_maintenance": false} lago.ssh: DEBUG: start task:1d1feb20-4955-4278-a7c2-31474375c7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1d1feb20-4955-4278-a7c2-31474375c7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b44fdf34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b44fdf34 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b44fdf34 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Mon Oct 1 17:44:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4180 (Mon Oct 1 17:44:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e129b4e8", "local_conf_timestamp": 4180, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4171 (Mon Oct 1 17:44:38 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4171 (Mon Oct 1 17:44:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f959e6cb", "local_conf_timestamp": 4171, "host-ts": 4171}, "global_maintenance": false} lago.ssh: DEBUG: start task:8905cbc0-83f0-483b-b912-4b5fd2938d60:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8905cbc0-83f0-483b-b912-4b5fd2938d60:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b74c5c58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b74c5c58 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b74c5c58 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Mon Oct 1 17:44:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4189 (Mon Oct 1 17:44:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Mon Oct 1 17:44:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Mon Oct 1 17:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5fb7b242", "local_conf_timestamp": 4182, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:58e67aef-c44d-4daf-8755-81373b6f625c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:58e67aef-c44d-4daf-8755-81373b6f625c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ba7acc98 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ba7acc98 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ba7acc98 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Mon Oct 1 17:44:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4189 (Mon Oct 1 17:44:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Mon Oct 1 17:44:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Mon Oct 1 17:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5fb7b242", "local_conf_timestamp": 4182, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:a1206639-871a-4f9d-a6ff-21c63f0c773c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a1206639-871a-4f9d-a6ff-21c63f0c773c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bd7a07ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bd7a07ec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bd7a07ec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Mon Oct 1 17:45:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Mon Oct 1 17:45:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Mon Oct 1 17:44:57 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4191 (Mon Oct 1 17:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71dcc473", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:9278ba90-85e0-424e-98de-6d3107fe706a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9278ba90-85e0-424e-98de-6d3107fe706a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c0716300 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c0716300 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c0716300 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Mon Oct 1 17:45:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Mon Oct 1 17:45:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Mon Oct 1 17:44:57 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4191 (Mon Oct 1 17:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71dcc473", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:4614fc93-a3c6-4b95-bd93-8853d27fc4eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4614fc93-a3c6-4b95-bd93-8853d27fc4eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c36df4ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c36df4ec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c36df4ec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Mon Oct 1 17:45:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Mon Oct 1 17:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Mon Oct 1 17:45:08 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4201 (Mon Oct 1 17:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:918c5d35-7651-4262-9c32-b5d41f8e802d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:918c5d35-7651-4262-9c32-b5d41f8e802d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c67c72e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c67c72e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c67c72e4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Mon Oct 1 17:45:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Mon Oct 1 17:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Mon Oct 1 17:45:08 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4201 (Mon Oct 1 17:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:a88f0696-6402-422a-92f3-d2e4f60f26e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a88f0696-6402-422a-92f3-d2e4f60f26e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c9773272 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c9773272 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c9773272 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Mon Oct 1 17:45:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4219 (Mon Oct 1 17:45:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "331de7a0", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Mon Oct 1 17:45:18 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4211 (Mon Oct 1 17:45:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:e684349c-d93e-4397-bff7-2ead9f827916:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e684349c-d93e-4397-bff7-2ead9f827916:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ca8e1c34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ca8e1c34 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ca8e1c34 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Mon Oct 1 17:45:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4219 (Mon Oct 1 17:45:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "331de7a0", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Mon Oct 1 17:45:18 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4211 (Mon Oct 1 17:45:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 40, in _wait_for_engine_migration lambda: _check_migration_state(host, state) is False File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:3becef3a-a3bc-451d-a134-b37387ff9a6c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:3becef3a-a3bc-451d-a134-b37387ff9a6c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running a167d066 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a167d066 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command a167d066 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4148 (Mon Oct 1 17:44:15 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4148 (Mon Oct 1 17:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7270d4f8", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4151 (Mon Oct 1 17:44:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4151 (Mon Oct 1 17:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "86e92d59", "local_conf_timestamp": 4151, "host-ts": 4151}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6908cdbc-d88c-4fe6-809f-dec0c42c43dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6908cdbc-d88c-4fe6-809f-dec0c42c43dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a2a75e24 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command a2a75e24 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:a6239f1a-9d61-4224-8d37-45d222f2927f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a6239f1a-9d61-4224-8d37-45d222f2927f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a3d02826 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a3d02826 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a3d02826 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4159 (Mon Oct 1 17:44:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4159 (Mon Oct 1 17:44:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3c3533c5", "local_conf_timestamp": 4159, "host-ts": 4159}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4151 (Mon Oct 1 17:44:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4151 (Mon Oct 1 17:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "86e92d59", "local_conf_timestamp": 4151, "host-ts": 4151}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b2d0f62d-f55e-4c2b-a2e2-e1038dd1481f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b2d0f62d-f55e-4c2b-a2e2-e1038dd1481f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ab0e8de4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ab0e8de4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ab0e8de4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Mon Oct 1 17:44:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4169 (Mon Oct 1 17:44:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b15e2e41", "local_conf_timestamp": 4169, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4161 (Mon Oct 1 17:44:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4161 (Mon Oct 1 17:44:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8a222c56", "local_conf_timestamp": 4161, "host-ts": 4161}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e1de5fb6-a808-4e10-973e-65371f552b71:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e1de5fb6-a808-4e10-973e-65371f552b71:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ae15c138 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ae15c138 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ae15c138 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Mon Oct 1 17:44:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4169 (Mon Oct 1 17:44:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b15e2e41", "local_conf_timestamp": 4169, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4161 (Mon Oct 1 17:44:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4161 (Mon Oct 1 17:44:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8a222c56", "local_conf_timestamp": 4161, "host-ts": 4161}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:459dcf92-4a1d-44a8-8c57-37c4a8d94644:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:459dcf92-4a1d-44a8-8c57-37c4a8d94644:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b144dc18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b144dc18 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b144dc18 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Mon Oct 1 17:44:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4180 (Mon Oct 1 17:44:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e129b4e8", "local_conf_timestamp": 4180, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4171 (Mon Oct 1 17:44:38 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4171 (Mon Oct 1 17:44:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f959e6cb", "local_conf_timestamp": 4171, "host-ts": 4171}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1d1feb20-4955-4278-a7c2-31474375c7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1d1feb20-4955-4278-a7c2-31474375c7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b44fdf34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b44fdf34 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b44fdf34 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Mon Oct 1 17:44:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4180 (Mon Oct 1 17:44:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e129b4e8", "local_conf_timestamp": 4180, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4171 (Mon Oct 1 17:44:38 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4171 (Mon Oct 1 17:44:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f959e6cb", "local_conf_timestamp": 4171, "host-ts": 4171}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8905cbc0-83f0-483b-b912-4b5fd2938d60:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8905cbc0-83f0-483b-b912-4b5fd2938d60:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b74c5c58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b74c5c58 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b74c5c58 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Mon Oct 1 17:44:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4189 (Mon Oct 1 17:44:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Mon Oct 1 17:44:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Mon Oct 1 17:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5fb7b242", "local_conf_timestamp": 4182, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:58e67aef-c44d-4daf-8755-81373b6f625c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:58e67aef-c44d-4daf-8755-81373b6f625c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ba7acc98 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ba7acc98 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ba7acc98 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Mon Oct 1 17:44:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4189 (Mon Oct 1 17:44:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Mon Oct 1 17:44:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Mon Oct 1 17:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5fb7b242", "local_conf_timestamp": 4182, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a1206639-871a-4f9d-a6ff-21c63f0c773c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a1206639-871a-4f9d-a6ff-21c63f0c773c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bd7a07ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bd7a07ec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bd7a07ec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Mon Oct 1 17:45:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Mon Oct 1 17:45:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Mon Oct 1 17:44:57 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4191 (Mon Oct 1 17:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71dcc473", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9278ba90-85e0-424e-98de-6d3107fe706a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9278ba90-85e0-424e-98de-6d3107fe706a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c0716300 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c0716300 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c0716300 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Mon Oct 1 17:45:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Mon Oct 1 17:45:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Mon Oct 1 17:44:57 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4191 (Mon Oct 1 17:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71dcc473", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4614fc93-a3c6-4b95-bd93-8853d27fc4eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4614fc93-a3c6-4b95-bd93-8853d27fc4eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c36df4ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c36df4ec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c36df4ec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Mon Oct 1 17:45:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Mon Oct 1 17:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Mon Oct 1 17:45:08 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4201 (Mon Oct 1 17:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:918c5d35-7651-4262-9c32-b5d41f8e802d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:918c5d35-7651-4262-9c32-b5d41f8e802d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c67c72e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c67c72e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c67c72e4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Mon Oct 1 17:45:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Mon Oct 1 17:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Mon Oct 1 17:45:08 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4201 (Mon Oct 1 17:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a88f0696-6402-422a-92f3-d2e4f60f26e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a88f0696-6402-422a-92f3-d2e4f60f26e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c9773272 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c9773272 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c9773272 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Mon Oct 1 17:45:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4219 (Mon Oct 1 17:45:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "331de7a0", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Mon Oct 1 17:45:18 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4211 (Mon Oct 1 17:45:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e684349c-d93e-4397-bff7-2ead9f827916:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e684349c-d93e-4397-bff7-2ead9f827916:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ca8e1c34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ca8e1c34 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ca8e1c34 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Mon Oct 1 17:45:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4219 (Mon Oct 1 17:45:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "331de7a0", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Mon Oct 1 17:45:18 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4211 (Mon Oct 1 17:45:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/363/ Build Number: 363 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- No tests ran.
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/364/ Build Number: 364 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.clear_global_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 76, in clear_global_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/365/ Build Number: 365 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/366/ Build Number: 366 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/367/ Build Number: 367 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- No tests ran.
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/368/ Build Number: 368 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:ad429a0a-7f61-401b-91e0-821ce9ef9e9a:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:ad429a0a-7f61-401b-91e0-821ce9ef9e9a:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 7283486e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7283486e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 7283486e on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4131 (Wed Oct 3 17:40:25 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4131 (Wed Oct 3 17:40:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "31a27c1f", "local_conf_timestamp": 4131, "host-ts": 4131}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4139 (Wed Oct 3 17:40:33 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4139 (Wed Oct 3 17:40:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "aa91bef1", "local_conf_timestamp": 4139, "host-ts": 4139}, "global_maintenance": false} lago.ssh: DEBUG: start task:80a7fc53-5ec1-4364-9b72-70d6d3df80f9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:80a7fc53-5ec1-4364-9b72-70d6d3df80f9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7391cd66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 7391cd66 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:f0b8cc24-38e4-49b2-bbb9-2bae4088dee3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f0b8cc24-38e4-49b2-bbb9-2bae4088dee3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 749f6362 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 749f6362 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 749f6362 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4141 (Wed Oct 3 17:40:35 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4141 (Wed Oct 3 17:40:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4905b661", "local_conf_timestamp": 4141, "host-ts": 4141}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4149 (Wed Oct 3 17:40:43 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4149 (Wed Oct 3 17:40:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b74143ea", "local_conf_timestamp": 4149, "host-ts": 4149}, "global_maintenance": false} lago.ssh: DEBUG: start task:724657f7-c34b-49c3-b833-d51b912ce656:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:724657f7-c34b-49c3-b833-d51b912ce656:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7c5adeb0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7c5adeb0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7c5adeb0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Wed Oct 3 17:40:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4163 (Wed Oct 3 17:40:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "44b5dfea", "local_conf_timestamp": 4163, "host-ts": 4162}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4159 (Wed Oct 3 17:40:52 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4159 (Wed Oct 3 17:40:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false} lago.ssh: DEBUG: start task:c2acab90-936a-40a2-9940-cab3be31a4cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c2acab90-936a-40a2-9940-cab3be31a4cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7fe4458a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7fe4458a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7fe4458a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Wed Oct 3 17:40:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4163 (Wed Oct 3 17:40:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "44b5dfea", "local_conf_timestamp": 4163, "host-ts": 4162}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4159 (Wed Oct 3 17:40:52 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4159 (Wed Oct 3 17:40:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false} lago.ssh: DEBUG: start task:5907d8d5-612c-4044-aa74-e4c6ae1f3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5907d8d5-612c-4044-aa74-e4c6ae1f3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 82dcc136 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 82dcc136 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 82dcc136 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Wed Oct 3 17:41:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4172 (Wed Oct 3 17:41:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9d81d1be", "local_conf_timestamp": 4172, "host-ts": 4172}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Wed Oct 3 17:41:03 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4169 (Wed Oct 3 17:41:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8f18878", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false} lago.ssh: DEBUG: start task:204bc533-318b-400f-926e-32c3af737048:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:204bc533-318b-400f-926e-32c3af737048:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 85be69ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 85be69ea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 85be69ea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Wed Oct 3 17:41:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4172 (Wed Oct 3 17:41:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9d81d1be", "local_conf_timestamp": 4172, "host-ts": 4172}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Wed Oct 3 17:41:03 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4169 (Wed Oct 3 17:41:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8f18878", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false} lago.ssh: DEBUG: start task:e313ecc3-b0dd-4591-a3a0-f25a86c2b098:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e313ecc3-b0dd-4591-a3a0-f25a86c2b098:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 88ae07fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 88ae07fa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 88ae07fa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Wed Oct 3 17:41:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4182 (Wed Oct 3 17:41:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "103480a5", "local_conf_timestamp": 4182, "host-ts": 4182}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Wed Oct 3 17:41:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4179 (Wed Oct 3 17:41:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ccb7777d", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false} lago.ssh: DEBUG: start task:f4bcb6e0-09f1-4b3c-a2ea-ab9d86199a63:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f4bcb6e0-09f1-4b3c-a2ea-ab9d86199a63:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8b963bae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8b963bae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8b963bae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Wed Oct 3 17:41:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4182 (Wed Oct 3 17:41:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "103480a5", "local_conf_timestamp": 4182, "host-ts": 4182}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Wed Oct 3 17:41:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4179 (Wed Oct 3 17:41:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ccb7777d", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false} lago.ssh: DEBUG: start task:83dfa644-335e-49c5-a100-8ee4821e09af:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:83dfa644-335e-49c5-a100-8ee4821e09af:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8e705cba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8e705cba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8e705cba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Wed Oct 3 17:41:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4193 (Wed Oct 3 17:41:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9008ef1", "local_conf_timestamp": 4193, "host-ts": 4192}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Wed Oct 3 17:41:23 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4189 (Wed Oct 3 17:41:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4074aac0", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false} lago.ssh: DEBUG: start task:57a7826c-e1ca-47ff-abfe-cfdfe72e01c8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:57a7826c-e1ca-47ff-abfe-cfdfe72e01c8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 914f0756 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 914f0756 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 914f0756 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Wed Oct 3 17:41:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4193 (Wed Oct 3 17:41:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9008ef1", "local_conf_timestamp": 4193, "host-ts": 4192}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Wed Oct 3 17:41:23 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4189 (Wed Oct 3 17:41:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4074aac0", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false} lago.ssh: DEBUG: start task:39240513-c05e-49cd-905b-86b105d22f90:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:39240513-c05e-49cd-905b-86b105d22f90:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 94a4b8ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 94a4b8ec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 94a4b8ec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Wed Oct 3 17:41:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4203 (Wed Oct 3 17:41:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1de35c7c", "local_conf_timestamp": 4203, "host-ts": 4203}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Wed Oct 3 17:41:32 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4199 (Wed Oct 3 17:41:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "443255c5", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:4be751b4-e338-469e-b842-9543aa31ba0b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4be751b4-e338-469e-b842-9543aa31ba0b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 97ae3cac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 97ae3cac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 97ae3cac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Wed Oct 3 17:41:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4203 (Wed Oct 3 17:41:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1de35c7c", "local_conf_timestamp": 4203, "host-ts": 4203}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Wed Oct 3 17:41:32 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4199 (Wed Oct 3 17:41:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "443255c5", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:0b09e5bc-0a64-4ac4-87f0-e9bca3b8ebec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0b09e5bc-0a64-4ac4-87f0-e9bca3b8ebec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9ab7c8e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9ab7c8e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9ab7c8e6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Wed Oct 3 17:41:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4214 (Wed Oct 3 17:41:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "005b8435", "local_conf_timestamp": 4214, "host-ts": 4213}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Wed Oct 3 17:41:43 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4209 (Wed Oct 3 17:41:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:37f59e5c-ecc2-4a03-80ce-250ee5d336d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:37f59e5c-ecc2-4a03-80ce-250ee5d336d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9bc09dda on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9bc09dda on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9bc09dda on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Wed Oct 3 17:41:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4214 (Wed Oct 3 17:41:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "005b8435", "local_conf_timestamp": 4214, "host-ts": 4213}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Wed Oct 3 17:41:43 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4209 (Wed Oct 3 17:41:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:284c803c-0d50-4d9b-9dd7-38532643d780:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:284c803c-0d50-4d9b-9dd7-38532643d780:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a2c70c18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a2c70c18 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a2c70c18 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Wed Oct 3 17:41:58 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4224 (Wed Oct 3 17:41:58 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "52afce4b", "local_conf_timestamp": 4224, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Wed Oct 3 17:41:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4219 (Wed Oct 3 17:41:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d69cf73e", "local_conf_timestamp": 4219, "host-ts": 4219}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:ad429a0a-7f61-401b-91e0-821ce9ef9e9a:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:ad429a0a-7f61-401b-91e0-821ce9ef9e9a:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 7283486e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7283486e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 7283486e on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4131 (Wed Oct 3 17:40:25 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4131 (Wed Oct 3 17:40:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "31a27c1f", "local_conf_timestamp": 4131, "host-ts": 4131}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4139 (Wed Oct 3 17:40:33 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4139 (Wed Oct 3 17:40:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "aa91bef1", "local_conf_timestamp": 4139, "host-ts": 4139}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:80a7fc53-5ec1-4364-9b72-70d6d3df80f9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:80a7fc53-5ec1-4364-9b72-70d6d3df80f9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7391cd66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 7391cd66 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:f0b8cc24-38e4-49b2-bbb9-2bae4088dee3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f0b8cc24-38e4-49b2-bbb9-2bae4088dee3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 749f6362 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 749f6362 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 749f6362 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4141 (Wed Oct 3 17:40:35 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4141 (Wed Oct 3 17:40:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4905b661", "local_conf_timestamp": 4141, "host-ts": 4141}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4149 (Wed Oct 3 17:40:43 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4149 (Wed Oct 3 17:40:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b74143ea", "local_conf_timestamp": 4149, "host-ts": 4149}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:724657f7-c34b-49c3-b833-d51b912ce656:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:724657f7-c34b-49c3-b833-d51b912ce656:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7c5adeb0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7c5adeb0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7c5adeb0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Wed Oct 3 17:40:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4163 (Wed Oct 3 17:40:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "44b5dfea", "local_conf_timestamp": 4163, "host-ts": 4162}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4159 (Wed Oct 3 17:40:52 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4159 (Wed Oct 3 17:40:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c2acab90-936a-40a2-9940-cab3be31a4cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c2acab90-936a-40a2-9940-cab3be31a4cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7fe4458a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7fe4458a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7fe4458a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Wed Oct 3 17:40:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4163 (Wed Oct 3 17:40:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "44b5dfea", "local_conf_timestamp": 4163, "host-ts": 4162}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4159 (Wed Oct 3 17:40:52 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4159 (Wed Oct 3 17:40:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5907d8d5-612c-4044-aa74-e4c6ae1f3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5907d8d5-612c-4044-aa74-e4c6ae1f3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 82dcc136 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 82dcc136 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 82dcc136 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Wed Oct 3 17:41:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4172 (Wed Oct 3 17:41:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9d81d1be", "local_conf_timestamp": 4172, "host-ts": 4172}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Wed Oct 3 17:41:03 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4169 (Wed Oct 3 17:41:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8f18878", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:204bc533-318b-400f-926e-32c3af737048:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:204bc533-318b-400f-926e-32c3af737048:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 85be69ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 85be69ea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 85be69ea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Wed Oct 3 17:41:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4172 (Wed Oct 3 17:41:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9d81d1be", "local_conf_timestamp": 4172, "host-ts": 4172}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Wed Oct 3 17:41:03 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4169 (Wed Oct 3 17:41:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8f18878", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e313ecc3-b0dd-4591-a3a0-f25a86c2b098:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e313ecc3-b0dd-4591-a3a0-f25a86c2b098:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 88ae07fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 88ae07fa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 88ae07fa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Wed Oct 3 17:41:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4182 (Wed Oct 3 17:41:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "103480a5", "local_conf_timestamp": 4182, "host-ts": 4182}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Wed Oct 3 17:41:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4179 (Wed Oct 3 17:41:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ccb7777d", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f4bcb6e0-09f1-4b3c-a2ea-ab9d86199a63:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f4bcb6e0-09f1-4b3c-a2ea-ab9d86199a63:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8b963bae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8b963bae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8b963bae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Wed Oct 3 17:41:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4182 (Wed Oct 3 17:41:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "103480a5", "local_conf_timestamp": 4182, "host-ts": 4182}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Wed Oct 3 17:41:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4179 (Wed Oct 3 17:41:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ccb7777d", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:83dfa644-335e-49c5-a100-8ee4821e09af:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:83dfa644-335e-49c5-a100-8ee4821e09af:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8e705cba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8e705cba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8e705cba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Wed Oct 3 17:41:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4193 (Wed Oct 3 17:41:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9008ef1", "local_conf_timestamp": 4193, "host-ts": 4192}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Wed Oct 3 17:41:23 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4189 (Wed Oct 3 17:41:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4074aac0", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:57a7826c-e1ca-47ff-abfe-cfdfe72e01c8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:57a7826c-e1ca-47ff-abfe-cfdfe72e01c8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 914f0756 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 914f0756 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 914f0756 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Wed Oct 3 17:41:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4193 (Wed Oct 3 17:41:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9008ef1", "local_conf_timestamp": 4193, "host-ts": 4192}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Wed Oct 3 17:41:23 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4189 (Wed Oct 3 17:41:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4074aac0", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:39240513-c05e-49cd-905b-86b105d22f90:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:39240513-c05e-49cd-905b-86b105d22f90:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 94a4b8ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 94a4b8ec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 94a4b8ec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Wed Oct 3 17:41:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4203 (Wed Oct 3 17:41:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1de35c7c", "local_conf_timestamp": 4203, "host-ts": 4203}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Wed Oct 3 17:41:32 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4199 (Wed Oct 3 17:41:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "443255c5", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4be751b4-e338-469e-b842-9543aa31ba0b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4be751b4-e338-469e-b842-9543aa31ba0b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 97ae3cac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 97ae3cac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 97ae3cac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Wed Oct 3 17:41:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4203 (Wed Oct 3 17:41:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1de35c7c", "local_conf_timestamp": 4203, "host-ts": 4203}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Wed Oct 3 17:41:32 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4199 (Wed Oct 3 17:41:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "443255c5", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0b09e5bc-0a64-4ac4-87f0-e9bca3b8ebec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0b09e5bc-0a64-4ac4-87f0-e9bca3b8ebec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9ab7c8e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9ab7c8e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9ab7c8e6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Wed Oct 3 17:41:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4214 (Wed Oct 3 17:41:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "005b8435", "local_conf_timestamp": 4214, "host-ts": 4213}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Wed Oct 3 17:41:43 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4209 (Wed Oct 3 17:41:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:37f59e5c-ecc2-4a03-80ce-250ee5d336d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:37f59e5c-ecc2-4a03-80ce-250ee5d336d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9bc09dda on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9bc09dda on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9bc09dda on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Wed Oct 3 17:41:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4214 (Wed Oct 3 17:41:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "005b8435", "local_conf_timestamp": 4214, "host-ts": 4213}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Wed Oct 3 17:41:43 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4209 (Wed Oct 3 17:41:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:284c803c-0d50-4d9b-9dd7-38532643d780:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:284c803c-0d50-4d9b-9dd7-38532643d780:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a2c70c18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a2c70c18 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a2c70c18 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Wed Oct 3 17:41:58 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4224 (Wed Oct 3 17:41:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "52afce4b", "local_conf_timestamp": 4224, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Wed Oct 3 17:41:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4219 (Wed Oct 3 17:41:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d69cf73e", "local_conf_timestamp": 4219, "host-ts": 4219}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/369/ Build Number: 369 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:6ba7ad28-a98b-4b15-a9ae-75f535d1afa8:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:6ba7ad28-a98b-4b15-a9ae-75f535d1afa8:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running e9c34968 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e9c34968 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command e9c34968 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4149 (Thu Oct 4 01:43:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4150 (Thu Oct 4 01:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a3faad33", "local_conf_timestamp": 4150, "host-ts": 4149}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4153 (Thu Oct 4 01:43:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4154 (Thu Oct 4 01:43:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f84e9fb7", "local_conf_timestamp": 4154, "host-ts": 4153}, "global_maintenance": false} lago.ssh: DEBUG: start task:a3bf47be-5b0d-461c-ba11-b13d06b9e274:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a3bf47be-5b0d-461c-ba11-b13d06b9e274:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running eadceaa2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command eadceaa2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:9b810606-6844-4000-b822-0fd7399002ea:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9b810606-6844-4000-b822-0fd7399002ea:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ebec683c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ebec683c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ebec683c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4160 (Thu Oct 4 01:43:38 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4160 (Thu Oct 4 01:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9bf7c070", "local_conf_timestamp": 4160, "host-ts": 4160}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4153 (Thu Oct 4 01:43:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4154 (Thu Oct 4 01:43:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f84e9fb7", "local_conf_timestamp": 4154, "host-ts": 4153}, "global_maintenance": false} lago.ssh: DEBUG: start task:79bdb3a7-32ab-46f8-b88f-38b40621ca24:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:79bdb3a7-32ab-46f8-b88f-38b40621ca24:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f3699210 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f3699210 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f3699210 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4170 (Thu Oct 4 01:43:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4170 (Thu Oct 4 01:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "64ffd763", "local_conf_timestamp": 4170, "host-ts": 4170}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4163 (Thu Oct 4 01:43:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4163 (Thu Oct 4 01:43:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ae10b1b", "local_conf_timestamp": 4163, "host-ts": 4163}, "global_maintenance": false} lago.ssh: DEBUG: start task:a6e31e23-6868-4da5-a20c-2450a686ea9e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a6e31e23-6868-4da5-a20c-2450a686ea9e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f7a72248 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f7a72248 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f7a72248 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Thu Oct 4 01:43:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4180 (Thu Oct 4 01:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0fec9734", "local_conf_timestamp": 4180, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4173 (Thu Oct 4 01:43:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4173 (Thu Oct 4 01:43:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false} lago.ssh: DEBUG: start task:5f4ddf16-dd86-4fd1-870d-b8caf847e991:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5f4ddf16-dd86-4fd1-870d-b8caf847e991:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running faf35868 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command faf35868 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command faf35868 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Thu Oct 4 01:43:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4180 (Thu Oct 4 01:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0fec9734", "local_conf_timestamp": 4180, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4173 (Thu Oct 4 01:43:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4173 (Thu Oct 4 01:43:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false} lago.ssh: DEBUG: start task:6373533b-82a8-40a6-aabd-6481e1becd02:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6373533b-82a8-40a6-aabd-6481e1becd02:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fdee8c86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fdee8c86 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fdee8c86 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Thu Oct 4 01:44:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4190 (Thu Oct 4 01:44:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4183 (Thu Oct 4 01:44:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4183 (Thu Oct 4 01:44:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false} lago.ssh: DEBUG: start task:fcafef7f-412f-4a6f-8bc0-b15f92747fe5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fcafef7f-412f-4a6f-8bc0-b15f92747fe5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 01080da2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 01080da2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 01080da2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Thu Oct 4 01:44:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4190 (Thu Oct 4 01:44:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4183 (Thu Oct 4 01:44:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4183 (Thu Oct 4 01:44:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false} lago.ssh: DEBUG: start task:316d5912-fd14-4d12-b2ef-33485b0de696:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:316d5912-fd14-4d12-b2ef-33485b0de696:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 04009f1a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 04009f1a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 04009f1a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4200 (Thu Oct 4 01:44:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4200 (Thu Oct 4 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4193 (Thu Oct 4 01:44:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4194 (Thu Oct 4 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc5fae13", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false} lago.ssh: DEBUG: start task:ad1df116-1203-438f-9e1b-3bef4b806be2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ad1df116-1203-438f-9e1b-3bef4b806be2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 06fa9194 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 06fa9194 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 06fa9194 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4200 (Thu Oct 4 01:44:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4200 (Thu Oct 4 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4193 (Thu Oct 4 01:44:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4194 (Thu Oct 4 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc5fae13", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false} lago.ssh: DEBUG: start task:0291756c-ccf9-4857-a0c8-26bd528eea6b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0291756c-ccf9-4857-a0c8-26bd528eea6b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 09f2c75e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 09f2c75e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 09f2c75e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Thu Oct 4 01:44:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4211 (Thu Oct 4 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "395bcd51", "local_conf_timestamp": 4211, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Thu Oct 4 01:44:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4203 (Thu Oct 4 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:7572cdc3-1a7a-4fe7-b5cf-017ad11e0b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7572cdc3-1a7a-4fe7-b5cf-017ad11e0b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0ceb4328 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0ceb4328 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0ceb4328 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Thu Oct 4 01:44:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4211 (Thu Oct 4 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "395bcd51", "local_conf_timestamp": 4211, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Thu Oct 4 01:44:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4203 (Thu Oct 4 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:93e99f9a-efd7-4931-aaea-1b4518d5c66b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:93e99f9a-efd7-4931-aaea-1b4518d5c66b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0fee0722 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0fee0722 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0fee0722 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Thu Oct 4 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4221 (Thu Oct 4 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Thu Oct 4 01:44:31 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4213 (Thu Oct 4 01:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:3043729b-f952-43fb-8085-585686fda4d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3043729b-f952-43fb-8085-585686fda4d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1324ee60 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1324ee60 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1324ee60 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Thu Oct 4 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4221 (Thu Oct 4 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Thu Oct 4 01:44:31 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4213 (Thu Oct 4 01:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:5eb56981-caa9-46e2-953d-a0a6b3bd3be6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5eb56981-caa9-46e2-953d-a0a6b3bd3be6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 16205348 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 16205348 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 16205348 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Thu Oct 4 01:44:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4231 (Thu Oct 4 01:44:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d584ad7", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Thu Oct 4 01:44:41 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4223 (Thu Oct 4 01:44:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:19ace601-f2e9-46fc-be69-c4cea285e655:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:19ace601-f2e9-46fc-be69-c4cea285e655:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1743a162 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1743a162 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1743a162 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Thu Oct 4 01:44:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4231 (Thu Oct 4 01:44:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d584ad7", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Thu Oct 4 01:44:41 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4223 (Thu Oct 4 01:44:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:e615bed8-a2bb-409c-9b17-5dffa9bb0254:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e615bed8-a2bb-409c-9b17-5dffa9bb0254:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1e62ce00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1e62ce00 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1e62ce00 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Thu Oct 4 01:44:59 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4241 (Thu Oct 4 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "f564c4d4", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Thu Oct 4 01:44:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4233 (Thu Oct 4 01:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b8964c40", "local_conf_timestamp": 4233, "host-ts": 4234}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:6ba7ad28-a98b-4b15-a9ae-75f535d1afa8:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:6ba7ad28-a98b-4b15-a9ae-75f535d1afa8:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running e9c34968 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e9c34968 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command e9c34968 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4149 (Thu Oct 4 01:43:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4150 (Thu Oct 4 01:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a3faad33", "local_conf_timestamp": 4150, "host-ts": 4149}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4153 (Thu Oct 4 01:43:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4154 (Thu Oct 4 01:43:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f84e9fb7", "local_conf_timestamp": 4154, "host-ts": 4153}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a3bf47be-5b0d-461c-ba11-b13d06b9e274:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a3bf47be-5b0d-461c-ba11-b13d06b9e274:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running eadceaa2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command eadceaa2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:9b810606-6844-4000-b822-0fd7399002ea:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9b810606-6844-4000-b822-0fd7399002ea:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ebec683c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ebec683c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ebec683c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4160 (Thu Oct 4 01:43:38 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4160 (Thu Oct 4 01:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9bf7c070", "local_conf_timestamp": 4160, "host-ts": 4160}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4153 (Thu Oct 4 01:43:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4154 (Thu Oct 4 01:43:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f84e9fb7", "local_conf_timestamp": 4154, "host-ts": 4153}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:79bdb3a7-32ab-46f8-b88f-38b40621ca24:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:79bdb3a7-32ab-46f8-b88f-38b40621ca24:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f3699210 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f3699210 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f3699210 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4170 (Thu Oct 4 01:43:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4170 (Thu Oct 4 01:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "64ffd763", "local_conf_timestamp": 4170, "host-ts": 4170}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4163 (Thu Oct 4 01:43:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4163 (Thu Oct 4 01:43:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ae10b1b", "local_conf_timestamp": 4163, "host-ts": 4163}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a6e31e23-6868-4da5-a20c-2450a686ea9e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a6e31e23-6868-4da5-a20c-2450a686ea9e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f7a72248 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f7a72248 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f7a72248 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Thu Oct 4 01:43:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4180 (Thu Oct 4 01:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0fec9734", "local_conf_timestamp": 4180, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4173 (Thu Oct 4 01:43:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4173 (Thu Oct 4 01:43:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5f4ddf16-dd86-4fd1-870d-b8caf847e991:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5f4ddf16-dd86-4fd1-870d-b8caf847e991:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running faf35868 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command faf35868 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command faf35868 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Thu Oct 4 01:43:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4180 (Thu Oct 4 01:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0fec9734", "local_conf_timestamp": 4180, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4173 (Thu Oct 4 01:43:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4173 (Thu Oct 4 01:43:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ea7f41e", "local_conf_timestamp": 4173, "host-ts": 4173}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6373533b-82a8-40a6-aabd-6481e1becd02:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6373533b-82a8-40a6-aabd-6481e1becd02:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fdee8c86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fdee8c86 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fdee8c86 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Thu Oct 4 01:44:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4190 (Thu Oct 4 01:44:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4183 (Thu Oct 4 01:44:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4183 (Thu Oct 4 01:44:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fcafef7f-412f-4a6f-8bc0-b15f92747fe5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fcafef7f-412f-4a6f-8bc0-b15f92747fe5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 01080da2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 01080da2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 01080da2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Thu Oct 4 01:44:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4190 (Thu Oct 4 01:44:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4183 (Thu Oct 4 01:44:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4183 (Thu Oct 4 01:44:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "267dc4b5", "local_conf_timestamp": 4183, "host-ts": 4183}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:316d5912-fd14-4d12-b2ef-33485b0de696:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:316d5912-fd14-4d12-b2ef-33485b0de696:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 04009f1a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 04009f1a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 04009f1a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4200 (Thu Oct 4 01:44:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4200 (Thu Oct 4 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4193 (Thu Oct 4 01:44:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4194 (Thu Oct 4 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc5fae13", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ad1df116-1203-438f-9e1b-3bef4b806be2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ad1df116-1203-438f-9e1b-3bef4b806be2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 06fa9194 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 06fa9194 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 06fa9194 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4200 (Thu Oct 4 01:44:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4200 (Thu Oct 4 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4193 (Thu Oct 4 01:44:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4194 (Thu Oct 4 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc5fae13", "local_conf_timestamp": 4194, "host-ts": 4193}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0291756c-ccf9-4857-a0c8-26bd528eea6b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0291756c-ccf9-4857-a0c8-26bd528eea6b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 09f2c75e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 09f2c75e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 09f2c75e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Thu Oct 4 01:44:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4211 (Thu Oct 4 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "395bcd51", "local_conf_timestamp": 4211, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Thu Oct 4 01:44:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4203 (Thu Oct 4 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7572cdc3-1a7a-4fe7-b5cf-017ad11e0b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7572cdc3-1a7a-4fe7-b5cf-017ad11e0b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0ceb4328 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0ceb4328 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0ceb4328 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Thu Oct 4 01:44:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4211 (Thu Oct 4 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "395bcd51", "local_conf_timestamp": 4211, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Thu Oct 4 01:44:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4203 (Thu Oct 4 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:93e99f9a-efd7-4931-aaea-1b4518d5c66b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:93e99f9a-efd7-4931-aaea-1b4518d5c66b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0fee0722 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0fee0722 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0fee0722 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Thu Oct 4 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4221 (Thu Oct 4 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Thu Oct 4 01:44:31 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4213 (Thu Oct 4 01:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3043729b-f952-43fb-8085-585686fda4d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3043729b-f952-43fb-8085-585686fda4d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1324ee60 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1324ee60 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1324ee60 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Thu Oct 4 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4221 (Thu Oct 4 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Thu Oct 4 01:44:31 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4213 (Thu Oct 4 01:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "239bcd86", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5eb56981-caa9-46e2-953d-a0a6b3bd3be6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5eb56981-caa9-46e2-953d-a0a6b3bd3be6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 16205348 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 16205348 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 16205348 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Thu Oct 4 01:44:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4231 (Thu Oct 4 01:44:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d584ad7", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Thu Oct 4 01:44:41 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4223 (Thu Oct 4 01:44:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:19ace601-f2e9-46fc-be69-c4cea285e655:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:19ace601-f2e9-46fc-be69-c4cea285e655:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1743a162 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1743a162 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1743a162 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Thu Oct 4 01:44:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4231 (Thu Oct 4 01:44:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d584ad7", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Thu Oct 4 01:44:41 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4223 (Thu Oct 4 01:44:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2f50cc89", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e615bed8-a2bb-409c-9b17-5dffa9bb0254:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e615bed8-a2bb-409c-9b17-5dffa9bb0254:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1e62ce00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1e62ce00 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1e62ce00 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Thu Oct 4 01:44:59 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4241 (Thu Oct 4 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "f564c4d4", "local_conf_timestamp": 4241, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Thu Oct 4 01:44:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4233 (Thu Oct 4 01:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b8964c40", "local_conf_timestamp": 4233, "host-ts": 4234}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/370/ Build Number: 370 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.restart_he_vm Error Message: 143 -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:f00620a7-648c-4102-80bc-305dfc8df78e:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:f00620a7-648c-4102-80bc-305dfc8df78e:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running aa8cf260 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aa8cf260 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command aa8cf260 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3242 (Thu Oct 4 09:41:15 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3242 (Thu Oct 4 09:41:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8b107fe5", "local_conf_timestamp": 3242, "host-ts": 3242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3246 (Thu Oct 4 09:41:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3246 (Thu Oct 4 09:41:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1fcb2337", "local_conf_timestamp": 3246, "host-ts": 3246}, "global_maintenance": true} root: INFO: * Shutting down HE VM on host: lago-he-node-ng-suite-4-2-host-0 lago.ssh: DEBUG: start task:ec93ce68-9d74-4565-953a-955d1eb0e8d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ec93ce68-9d74-4565-953a-955d1eb0e8d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running abab09a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-shutdown lago.ssh: DEBUG: Command abab09a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Command succeeded root: INFO: * Waiting for VM to be down... lago.ssh: DEBUG: start task:637d3a75-934a-403b-915a-ac54ccf78e16:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:637d3a75-934a-403b-915a-ac54ccf78e16:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ae5d3166 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ae5d3166 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ae5d3166 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3262 (Thu Oct 4 09:41:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3262 (Thu Oct 4 09:41:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "23914a5d", "local_conf_timestamp": 3262, "host-ts": 3262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3256 (Thu Oct 4 09:41:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3256 (Thu Oct 4 09:41:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b8ddc32", "local_conf_timestamp": 3256, "host-ts": 3256}, "global_maintenance": true} lago.ssh: DEBUG: start task:f3cc7ee7-b43b-46f1-b562-71fd5f30b603:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f3cc7ee7-b43b-46f1-b562-71fd5f30b603:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b614c996 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b614c996 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b614c996 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3272 (Thu Oct 4 09:41:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3272 (Thu Oct 4 09:41:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b515bfa9", "local_conf_timestamp": 3272, "host-ts": 3272}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3266 (Thu Oct 4 09:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3266 (Thu Oct 4 09:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1746dd3d", "local_conf_timestamp": 3266, "host-ts": 3266}, "global_maintenance": true} root: INFO: * VM is down. root: INFO: * Stopping services... lago.ssh: DEBUG: start task:e3ace6d9-9908-46d5-99cc-941775ff753d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e3ace6d9-9908-46d5-99cc-941775ff753d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b71f7034 on lago-he-node-ng-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command b71f7034 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Starting services... lago.ssh: DEBUG: start task:06f7c3be-084f-41e8-9b7e-7fd8fd888104:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:06f7c3be-084f-41e8-9b7e-7fd8fd888104:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c0e5d356 on lago-he-node-ng-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command c0e5d356 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for agent to be ready... lago.ssh: DEBUG: start task:1be0a2bc-9f9b-41bc-a071-bfd9fdba671b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1be0a2bc-9f9b-41bc-a071-bfd9fdba671b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c44de510 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command c44de510 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command c44de510 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:373c4570-500d-4ce6-927b-bb6123864318:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:373c4570-500d-4ce6-927b-bb6123864318:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb39b75a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command cb39b75a on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command cb39b75a on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:37a4e08d-5b8c-483b-933c-dce619e1dedc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:37a4e08d-5b8c-483b-933c-dce619e1dedc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cdf4d326 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command cdf4d326 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command cdf4d326 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:d521aeb6-5fcf-47c2-9a71-9a780446b6f8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d521aeb6-5fcf-47c2-9a71-9a780446b6f8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d09c5e6e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command d09c5e6e on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command d09c5e6e on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:b6cf3caa-8870-4f9c-b7aa-0efc0d342232:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b6cf3caa-8870-4f9c-b7aa-0efc0d342232:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d347692e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command d347692e on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command d347692e on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:418cf9ef-1ff0-4e84-907a-d8413925b236:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:418cf9ef-1ff0-4e84-907a-d8413925b236:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d5fb6a58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command d5fb6a58 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command d5fb6a58 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:dbd2db9e-0adf-4215-bfaf-f2d71b8673dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dbd2db9e-0adf-4215-bfaf-f2d71b8673dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d8b69a56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command d8b69a56 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command d8b69a56 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:9ca1ee48-a3eb-4211-8687-82c96b23ffc4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9ca1ee48-a3eb-4211-8687-82c96b23ffc4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running db62bfdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command db62bfdc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command db62bfdc on lago-he-node-ng-suite-4-2-host-0 output: !! Cluster is in GLOBAL MAINTENANCE mode !! --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : lago-he-node-ng-suite-4-2-host-0 Host ID : 1 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"} Score : 0 stopped : False Local maintenance : False crc32 : 5317f638 local_conf_timestamp : 3338 Host timestamp : 3337 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=3337 (Thu Oct 4 09:42:50 2018) host-id=1 score=0 vm_conf_refresh_time=3338 (Thu Oct 4 09:42:50 2018) conf_on_shared_storage=True maintenance=False state=ReinitializeFSM stopped=False --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : False Hostname : lago-he-node-ng-suite-4-2-host-1 Host ID : 2 Engine status : unknown stale-data Score : 3400 stopped : False Local maintenance : False crc32 : 72756c5c local_conf_timestamp : 3336 Host timestamp : 3336 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=3336 (Thu Oct 4 09:42:48 2018) host-id=2 score=3400 vm_conf_refresh_time=3336 (Thu Oct 4 09:42:48 2018) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False !! Cluster is in GLOBAL MAINTENANCE mode !! root: INFO: * Agent is ready. root: INFO: * Starting VM... lago.ssh: DEBUG: start task:256bf5d7-8dab-43b3-8663-aad3142b3931:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:256bf5d7-8dab-43b3-8663-aad3142b3931:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dc6fdb58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-start lago.ssh: DEBUG: Command dc6fdb58 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dc6fdb58 on lago-he-node-ng-suite-4-2-host-0 output: VM exists and is down, cleaning up and restarting root: INFO: * Command succeeded root: INFO: * Waiting for VM to be UP... lago.ssh: DEBUG: start task:81f8d6ed-81db-4fe1-bddd-4d14ae2fa5f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:81f8d6ed-81db-4fe1-bddd-4d14ae2fa5f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ded02ba0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ded02ba0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ded02ba0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3337 (Thu Oct 4 09:42:50 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=3338 (Thu Oct 4 09:42:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "5317f638", "local_conf_timestamp": 3338, "host-ts": 3337}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3336 (Thu Oct 4 09:42:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3336 (Thu Oct 4 09:42:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "72756c5c", "local_conf_timestamp": 3336, "host-ts": 3336}, "global_maintenance": true} lago.ssh: DEBUG: start task:59bc4945-9a1c-4406-b554-c17f80313d43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:59bc4945-9a1c-4406-b554-c17f80313d43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e5fe607c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e5fe607c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e5fe607c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3348 (Thu Oct 4 09:43:00 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3348 (Thu Oct 4 09:43:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "560c0311", "local_conf_timestamp": 3348, "host-ts": 3348}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3346 (Thu Oct 4 09:42:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3346 (Thu Oct 4 09:42:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6fa59147", "local_conf_timestamp": 3346, "host-ts": 3346}, "global_maintenance": true} root: INFO: * VM is UP. root: INFO: * Waiting for engine to start... lago.ssh: DEBUG: start task:42395fbb-6d4f-4700-9c2d-29b8f3c271b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:42395fbb-6d4f-4700-9c2d-29b8f3c271b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e719f840 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e719f840 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e719f840 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3348 (Thu Oct 4 09:43:00 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3348 (Thu Oct 4 09:43:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "560c0311", "local_conf_timestamp": 3348, "host-ts": 3348}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3346 (Thu Oct 4 09:42:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3346 (Thu Oct 4 09:42:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6fa59147", "local_conf_timestamp": 3346, "host-ts": 3346}, "global_maintenance": true} lago.ssh: DEBUG: start task:a894e3bb-c156-448c-95e9-6a2401b193a5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a894e3bb-c156-448c-95e9-6a2401b193a5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee354454 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ee354454 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ee354454 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3368 (Thu Oct 4 09:43:21 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3369 (Thu Oct 4 09:43:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e53497f", "local_conf_timestamp": 3369, "host-ts": 3368}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3366 (Thu Oct 4 09:43:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3366 (Thu Oct 4 09:43:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "67286f4d", "local_conf_timestamp": 3366, "host-ts": 3366}, "global_maintenance": true} lago.ssh: DEBUG: start task:2335e608-378b-412d-8e55-fc56f15b0046:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2335e608-378b-412d-8e55-fc56f15b0046:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f1b67954 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f1b67954 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f1b67954 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3368 (Thu Oct 4 09:43:21 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3369 (Thu Oct 4 09:43:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e53497f", "local_conf_timestamp": 3369, "host-ts": 3368}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3366 (Thu Oct 4 09:43:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3366 (Thu Oct 4 09:43:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "67286f4d", "local_conf_timestamp": 3366, "host-ts": 3366}, "global_maintenance": true} lago.ssh: DEBUG: start task:815ddfdf-c730-40b4-be0e-c849bacfc3b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:815ddfdf-c730-40b4-be0e-c849bacfc3b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f4cf656a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f4cf656a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f4cf656a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3378 (Thu Oct 4 09:43:30 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3378 (Thu Oct 4 09:43:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "16f84495", "local_conf_timestamp": 3378, "host-ts": 3378}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3376 (Thu Oct 4 09:43:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3376 (Thu Oct 4 09:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "636e9048", "local_conf_timestamp": 3376, "host-ts": 3376}, "global_maintenance": true} lago.ssh: DEBUG: start task:ea2f18ce-d614-466c-81f5-b758f3605b54:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ea2f18ce-d614-466c-81f5-b758f3605b54:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f828d642 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f828d642 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f828d642 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3378 (Thu Oct 4 09:43:30 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3378 (Thu Oct 4 09:43:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "16f84495", "local_conf_timestamp": 3378, "host-ts": 3378}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3376 (Thu Oct 4 09:43:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3376 (Thu Oct 4 09:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "636e9048", "local_conf_timestamp": 3376, "host-ts": 3376}, "global_maintenance": true} lago.ssh: DEBUG: start task:b6afb67b-a051-42dd-8b75-44020ce3ff10:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b6afb67b-a051-42dd-8b75-44020ce3ff10:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fb396cac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fb396cac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fb396cac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3388 (Thu Oct 4 09:43:41 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3388 (Thu Oct 4 09:43:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4fc508ca", "local_conf_timestamp": 3388, "host-ts": 3388}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3386 (Thu Oct 4 09:43:38 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3386 (Thu Oct 4 09:43:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5c89957b", "local_conf_timestamp": 3386, "host-ts": 3386}, "global_maintenance": true} lago.ssh: DEBUG: start task:20f3c2d4-19a5-40e0-8c97-7b3edb28aec3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:20f3c2d4-19a5-40e0-8c97-7b3edb28aec3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fe524544 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fe524544 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fe524544 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3398 (Thu Oct 4 09:43:50 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3398 (Thu Oct 4 09:43:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0e90c296", "local_conf_timestamp": 3398, "host-ts": 3398}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3396 (Thu Oct 4 09:43:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3396 (Thu Oct 4 09:43:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "58cf6a7e", "local_conf_timestamp": 3396, "host-ts": 3396}, "global_maintenance": true} lago.ssh: DEBUG: start task:9f48c0ef-ebcc-4a54-80d4-4d3e243de50e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9f48c0ef-ebcc-4a54-80d4-4d3e243de50e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 040a95b8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 040a95b8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 040a95b8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3398 (Thu Oct 4 09:43:50 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3398 (Thu Oct 4 09:43:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0e90c296", "local_conf_timestamp": 3398, "host-ts": 3398}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3396 (Thu Oct 4 09:43:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3396 (Thu Oct 4 09:43:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "58cf6a7e", "local_conf_timestamp": 3396, "host-ts": 3396}, "global_maintenance": true} lago.ssh: DEBUG: start task:429cd235-4fe5-4c16-b3b0-11f9e55560ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:429cd235-4fe5-4c16-b3b0-11f9e55560ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 076c265e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 076c265e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 076c265e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3408 (Thu Oct 4 09:44:01 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3409 (Thu Oct 4 09:44:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a95b9e55", "local_conf_timestamp": 3409, "host-ts": 3408}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3406 (Thu Oct 4 09:43:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3406 (Thu Oct 4 09:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f4c67442", "local_conf_timestamp": 3406, "host-ts": 3406}, "global_maintenance": true} lago.ssh: DEBUG: start task:2a141c53-21e1-4165-b46e-1cf570715f1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2a141c53-21e1-4165-b46e-1cf570715f1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0b16e942 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0b16e942 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0b16e942 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3408 (Thu Oct 4 09:44:01 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3409 (Thu Oct 4 09:44:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a95b9e55", "local_conf_timestamp": 3409, "host-ts": 3408}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3406 (Thu Oct 4 09:43:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3406 (Thu Oct 4 09:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f4c67442", "local_conf_timestamp": 3406, "host-ts": 3406}, "global_maintenance": true} lago.ssh: DEBUG: start task:e486af2a-d1e1-4b09-ae1b-4c295bac8e90:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e486af2a-d1e1-4b09-ae1b-4c295bac8e90:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0ec55f06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0ec55f06 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0ec55f06 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3418 (Thu Oct 4 09:44:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3418 (Thu Oct 4 09:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f09649f", "local_conf_timestamp": 3418, "host-ts": 3418}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3416 (Thu Oct 4 09:44:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3416 (Thu Oct 4 09:44:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f0808b47", "local_conf_timestamp": 3416, "host-ts": 3416}, "global_maintenance": true} lago.ssh: DEBUG: start task:f3668d90-a0c3-4f38-b486-1f5efa7f7c88:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f3668d90-a0c3-4f38-b486-1f5efa7f7c88:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1261b772 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1261b772 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1261b772 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3428 (Thu Oct 4 09:44:21 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3429 (Thu Oct 4 09:44:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2bf00aed", "local_conf_timestamp": 3429, "host-ts": 3428}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3426 (Thu Oct 4 09:44:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3426 (Thu Oct 4 09:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fc4b8a48", "local_conf_timestamp": 3426, "host-ts": 3426}, "global_maintenance": true} lago.ssh: DEBUG: start task:79c50e1a-ea5f-44e3-a92c-dbdad9dd29e2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:79c50e1a-ea5f-44e3-a92c-dbdad9dd29e2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 15ac24d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 15ac24d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 15ac24d0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3428 (Thu Oct 4 09:44:21 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3429 (Thu Oct 4 09:44:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2bf00aed", "local_conf_timestamp": 3429, "host-ts": 3428}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3426 (Thu Oct 4 09:44:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3426 (Thu Oct 4 09:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fc4b8a48", "local_conf_timestamp": 3426, "host-ts": 3426}, "global_maintenance": true} lago.ssh: DEBUG: start task:8ea9732b-ebbf-479e-a5f8-c33e269564f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8ea9732b-ebbf-479e-a5f8-c33e269564f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 18eb5c92 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 18eb5c92 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 18eb5c92 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3438 (Thu Oct 4 09:44:31 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3438 (Thu Oct 4 09:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1da2f027", "local_conf_timestamp": 3438, "host-ts": 3438}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3436 (Thu Oct 4 09:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3436 (Thu Oct 4 09:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f80d754d", "local_conf_timestamp": 3436, "host-ts": 3436}, "global_maintenance": true} lago.ssh: DEBUG: start task:f95ac86b-781d-4bc6-8a9a-a1a6224f395d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f95ac86b-781d-4bc6-8a9a-a1a6224f395d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1d18356a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1d18356a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1d18356a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3448 (Thu Oct 4 09:44:41 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3448 (Thu Oct 4 09:44:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "007a81f2", "local_conf_timestamp": 3448, "host-ts": 3448}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3446 (Thu Oct 4 09:44:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3447 (Thu Oct 4 09:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "92dab8c0", "local_conf_timestamp": 3447, "host-ts": 3446}, "global_maintenance": true} lago.ssh: DEBUG: start task:b0263ebe-2821-49e8-b0a7-2bfbb224a43b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b0263ebe-2821-49e8-b0a7-2bfbb224a43b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2179aae4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2179aae4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2179aae4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3448 (Thu Oct 4 09:44:41 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3448 (Thu Oct 4 09:44:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "007a81f2", "local_conf_timestamp": 3448, "host-ts": 3448}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3446 (Thu Oct 4 09:44:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3447 (Thu Oct 4 09:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "92dab8c0", "local_conf_timestamp": 3447, "host-ts": 3446}, "global_maintenance": true} lago.ssh: DEBUG: start task:456b4cca-60ce-4282-a4ea-a675560048e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:456b4cca-60ce-4282-a4ea-a675560048e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 253bc464 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 253bc464 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 253bc464 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3458 (Thu Oct 4 09:44:50 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3458 (Thu Oct 4 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "56af83fa", "local_conf_timestamp": 3458, "host-ts": 3458}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3456 (Thu Oct 4 09:44:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3456 (Thu Oct 4 09:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e19b7753", "local_conf_timestamp": 3456, "host-ts": 3456}, "global_maintenance": true} lago.ssh: DEBUG: start task:c63e8892-0155-4afc-a41d-f35f4eb94cd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c63e8892-0155-4afc-a41d-f35f4eb94cd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 28fdd538 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 28fdd538 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 28fdd538 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3468 (Thu Oct 4 09:45:01 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3468 (Thu Oct 4 09:45:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9551dd1e", "local_conf_timestamp": 3468, "host-ts": 3468}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3466 (Thu Oct 4 09:44:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3466 (Thu Oct 4 09:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed50765c", "local_conf_timestamp": 3466, "host-ts": 3466}, "global_maintenance": true} lago.ssh: DEBUG: start task:7297b9f6-3b34-4367-a8f8-f6fdd2e32ded:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7297b9f6-3b34-4367-a8f8-f6fdd2e32ded:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2d8da9ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2d8da9ca on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2d8da9ca on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3468 (Thu Oct 4 09:45:01 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3468 (Thu Oct 4 09:45:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9551dd1e", "local_conf_timestamp": 3468, "host-ts": 3468}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3466 (Thu Oct 4 09:44:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3466 (Thu Oct 4 09:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed50765c", "local_conf_timestamp": 3466, "host-ts": 3466}, "global_maintenance": true} lago.ssh: DEBUG: start task:d56805c8-ada4-4e0d-b735-22b02bcd3e91:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d56805c8-ada4-4e0d-b735-22b02bcd3e91:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 32f8f45a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 32f8f45a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 32f8f45a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3478 (Thu Oct 4 09:45:10 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3478 (Thu Oct 4 09:45:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d4041742", "local_conf_timestamp": 3478, "host-ts": 3478}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3476 (Thu Oct 4 09:45:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3476 (Thu Oct 4 09:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e9168959", "local_conf_timestamp": 3476, "host-ts": 3476}, "global_maintenance": true} lago.ssh: DEBUG: start task:f339adce-4d16-4509-a178-0f0c89fb9acc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f339adce-4d16-4509-a178-0f0c89fb9acc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 36c35c4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 36c35c4c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 36c35c4c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3488 (Thu Oct 4 09:45:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3489 (Thu Oct 4 09:45:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9e60e22", "local_conf_timestamp": 3489, "host-ts": 3488}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3486 (Thu Oct 4 09:45:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3486 (Thu Oct 4 09:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d6f18c6a", "local_conf_timestamp": 3486, "host-ts": 3486}, "global_maintenance": true} lago.ssh: DEBUG: start task:108b90d6-a84d-4089-8f0f-c5f017828dea:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:108b90d6-a84d-4089-8f0f-c5f017828dea:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3b44df52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3b44df52 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3b44df52 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3498 (Thu Oct 4 09:45:31 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3498 (Thu Oct 4 09:45:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "efb4f4e8", "local_conf_timestamp": 3498, "host-ts": 3498}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3496 (Thu Oct 4 09:45:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3496 (Thu Oct 4 09:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d2b7736f", "local_conf_timestamp": 3496, "host-ts": 3496}, "global_maintenance": true} lago.ssh: DEBUG: start task:705dfd9b-7751-4274-a837-69b6bc0c3228:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:705dfd9b-7751-4274-a837-69b6bc0c3228:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3f92ff80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 53, in restart_he_vm _wait_for_engine_health(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 185, in _wait_for_engine_health testlib.assert_true_within_long(lambda: any( File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda> for k, v in _get_he_status(host).items() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 122, in _get_he_status ret = host.ssh(["hosted-engine", "--vm-status", "--json"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 69, in ssh 'stderr': None File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 247, in drain_ssh_channel 0.1, File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:f00620a7-648c-4102-80bc-305dfc8df78e:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:f00620a7-648c-4102-80bc-305dfc8df78e:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running aa8cf260 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aa8cf260 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command aa8cf260 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3242 (Thu Oct 4 09:41:15 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3242 (Thu Oct 4 09:41:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8b107fe5", "local_conf_timestamp": 3242, "host-ts": 3242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3246 (Thu Oct 4 09:41:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3246 (Thu Oct 4 09:41:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1fcb2337", "local_conf_timestamp": 3246, "host-ts": 3246}, "global_maintenance": true}\n\nroot: INFO: * Shutting down HE VM on host: lago-he-node-ng-suite-4-2-host-0\nlago.ssh: DEBUG: start task:ec93ce68-9d74-4565-953a-955d1eb0e8d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ec93ce68-9d74-4565-953a-955d1eb0e8d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running abab09a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-shutdown\nlago.ssh: DEBUG: Command abab09a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be down...\nlago.ssh: DEBUG: start task:637d3a75-934a-403b-915a-ac54ccf78e16:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:637d3a75-934a-403b-915a-ac54ccf78e16:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ae5d3166 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ae5d3166 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ae5d3166 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3262 (Thu Oct 4 09:41:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3262 (Thu Oct 4 09:41:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "23914a5d", "local_conf_timestamp": 3262, "host-ts": 3262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3256 (Thu Oct 4 09:41:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3256 (Thu Oct 4 09:41:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b8ddc32", "local_conf_timestamp": 3256, "host-ts": 3256}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:f3cc7ee7-b43b-46f1-b562-71fd5f30b603:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f3cc7ee7-b43b-46f1-b562-71fd5f30b603:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b614c996 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b614c996 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b614c996 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3272 (Thu Oct 4 09:41:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3272 (Thu Oct 4 09:41:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b515bfa9", "local_conf_timestamp": 3272, "host-ts": 3272}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3266 (Thu Oct 4 09:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3266 (Thu Oct 4 09:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1746dd3d", "local_conf_timestamp": 3266, "host-ts": 3266}, "global_maintenance": true}\n\nroot: INFO: * VM is down.\nroot: INFO: * Stopping services...\nlago.ssh: DEBUG: start task:e3ace6d9-9908-46d5-99cc-941775ff753d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e3ace6d9-9908-46d5-99cc-941775ff753d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b71f7034 on lago-he-node-ng-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command b71f7034 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Starting services...\nlago.ssh: DEBUG: start task:06f7c3be-084f-41e8-9b7e-7fd8fd888104:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:06f7c3be-084f-41e8-9b7e-7fd8fd888104:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c0e5d356 on lago-he-node-ng-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command c0e5d356 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for agent to be ready...\nlago.ssh: DEBUG: start task:1be0a2bc-9f9b-41bc-a071-bfd9fdba671b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1be0a2bc-9f9b-41bc-a071-bfd9fdba671b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c44de510 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command c44de510 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command c44de510 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:373c4570-500d-4ce6-927b-bb6123864318:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:373c4570-500d-4ce6-927b-bb6123864318:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb39b75a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command cb39b75a on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command cb39b75a on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:37a4e08d-5b8c-483b-933c-dce619e1dedc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:37a4e08d-5b8c-483b-933c-dce619e1dedc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cdf4d326 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command cdf4d326 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command cdf4d326 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:d521aeb6-5fcf-47c2-9a71-9a780446b6f8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d521aeb6-5fcf-47c2-9a71-9a780446b6f8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d09c5e6e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command d09c5e6e on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command d09c5e6e on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:b6cf3caa-8870-4f9c-b7aa-0efc0d342232:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b6cf3caa-8870-4f9c-b7aa-0efc0d342232:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d347692e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command d347692e on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command d347692e on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:418cf9ef-1ff0-4e84-907a-d8413925b236:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:418cf9ef-1ff0-4e84-907a-d8413925b236:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d5fb6a58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command d5fb6a58 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command d5fb6a58 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:dbd2db9e-0adf-4215-bfaf-f2d71b8673dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dbd2db9e-0adf-4215-bfaf-f2d71b8673dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d8b69a56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command d8b69a56 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command d8b69a56 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:9ca1ee48-a3eb-4211-8687-82c96b23ffc4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9ca1ee48-a3eb-4211-8687-82c96b23ffc4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running db62bfdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command db62bfdc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command db62bfdc on lago-he-node-ng-suite-4-2-host-0 output:\n \n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\n\n--== Host 1 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date : True\nHostname : lago-he-node-ng-suite-4-2-host-0\nHost ID : 1\nEngine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}\nScore : 0\nstopped : False\nLocal maintenance : False\ncrc32 : 5317f638\nlocal_conf_timestamp : 3338\nHost timestamp : 3337\nExtra metadata (valid at timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=3337 (Thu Oct 4 09:42:50 2018)\n\thost-id=1\n\tscore=0\n\tvm_conf_refresh_time=3338 (Thu Oct 4 09:42:50 2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=ReinitializeFSM\n\tstopped=False\n\n\n--== Host 2 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date : False\nHostname : lago-he-node-ng-suite-4-2-host-1\nHost ID : 2\nEngine status : unknown stale-data\nScore : 3400\nstopped : False\nLocal maintenance : False\ncrc32 : 72756c5c\nlocal_conf_timestamp : 3336\nHost timestamp : 3336\nExtra metadata (valid at timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=3336 (Thu Oct 4 09:42:48 2018)\n\thost-id=2\n\tscore=3400\n\tvm_conf_refresh_time=3336 (Thu Oct 4 09:42:48 2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=GlobalMaintenance\n\tstopped=False\n\n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\nroot: INFO: * Agent is ready.\nroot: INFO: * Starting VM...\nlago.ssh: DEBUG: start task:256bf5d7-8dab-43b3-8663-aad3142b3931:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:256bf5d7-8dab-43b3-8663-aad3142b3931:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dc6fdb58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-start\nlago.ssh: DEBUG: Command dc6fdb58 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dc6fdb58 on lago-he-node-ng-suite-4-2-host-0 output:\n VM exists and is down, cleaning up and restarting\n\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be UP...\nlago.ssh: DEBUG: start task:81f8d6ed-81db-4fe1-bddd-4d14ae2fa5f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:81f8d6ed-81db-4fe1-bddd-4d14ae2fa5f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ded02ba0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ded02ba0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ded02ba0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3337 (Thu Oct 4 09:42:50 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=3338 (Thu Oct 4 09:42:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "5317f638", "local_conf_timestamp": 3338, "host-ts": 3337}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3336 (Thu Oct 4 09:42:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3336 (Thu Oct 4 09:42:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "72756c5c", "local_conf_timestamp": 3336, "host-ts": 3336}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:59bc4945-9a1c-4406-b554-c17f80313d43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:59bc4945-9a1c-4406-b554-c17f80313d43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e5fe607c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e5fe607c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e5fe607c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3348 (Thu Oct 4 09:43:00 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3348 (Thu Oct 4 09:43:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "560c0311", "local_conf_timestamp": 3348, "host-ts": 3348}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3346 (Thu Oct 4 09:42:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3346 (Thu Oct 4 09:42:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6fa59147", "local_conf_timestamp": 3346, "host-ts": 3346}, "global_maintenance": true}\n\nroot: INFO: * VM is UP.\nroot: INFO: * Waiting for engine to start...\nlago.ssh: DEBUG: start task:42395fbb-6d4f-4700-9c2d-29b8f3c271b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:42395fbb-6d4f-4700-9c2d-29b8f3c271b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e719f840 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e719f840 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e719f840 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3348 (Thu Oct 4 09:43:00 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3348 (Thu Oct 4 09:43:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "560c0311", "local_conf_timestamp": 3348, "host-ts": 3348}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3346 (Thu Oct 4 09:42:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3346 (Thu Oct 4 09:42:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6fa59147", "local_conf_timestamp": 3346, "host-ts": 3346}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:a894e3bb-c156-448c-95e9-6a2401b193a5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a894e3bb-c156-448c-95e9-6a2401b193a5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee354454 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ee354454 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ee354454 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3368 (Thu Oct 4 09:43:21 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3369 (Thu Oct 4 09:43:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e53497f", "local_conf_timestamp": 3369, "host-ts": 3368}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3366 (Thu Oct 4 09:43:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3366 (Thu Oct 4 09:43:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "67286f4d", "local_conf_timestamp": 3366, "host-ts": 3366}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:2335e608-378b-412d-8e55-fc56f15b0046:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2335e608-378b-412d-8e55-fc56f15b0046:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f1b67954 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f1b67954 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f1b67954 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3368 (Thu Oct 4 09:43:21 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3369 (Thu Oct 4 09:43:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e53497f", "local_conf_timestamp": 3369, "host-ts": 3368}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3366 (Thu Oct 4 09:43:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3366 (Thu Oct 4 09:43:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "67286f4d", "local_conf_timestamp": 3366, "host-ts": 3366}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:815ddfdf-c730-40b4-be0e-c849bacfc3b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:815ddfdf-c730-40b4-be0e-c849bacfc3b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f4cf656a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f4cf656a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f4cf656a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3378 (Thu Oct 4 09:43:30 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3378 (Thu Oct 4 09:43:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "16f84495", "local_conf_timestamp": 3378, "host-ts": 3378}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3376 (Thu Oct 4 09:43:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3376 (Thu Oct 4 09:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "636e9048", "local_conf_timestamp": 3376, "host-ts": 3376}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:ea2f18ce-d614-466c-81f5-b758f3605b54:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ea2f18ce-d614-466c-81f5-b758f3605b54:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f828d642 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f828d642 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f828d642 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3378 (Thu Oct 4 09:43:30 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3378 (Thu Oct 4 09:43:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "16f84495", "local_conf_timestamp": 3378, "host-ts": 3378}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3376 (Thu Oct 4 09:43:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3376 (Thu Oct 4 09:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "636e9048", "local_conf_timestamp": 3376, "host-ts": 3376}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:b6afb67b-a051-42dd-8b75-44020ce3ff10:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b6afb67b-a051-42dd-8b75-44020ce3ff10:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fb396cac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fb396cac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fb396cac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3388 (Thu Oct 4 09:43:41 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3388 (Thu Oct 4 09:43:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4fc508ca", "local_conf_timestamp": 3388, "host-ts": 3388}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3386 (Thu Oct 4 09:43:38 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3386 (Thu Oct 4 09:43:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5c89957b", "local_conf_timestamp": 3386, "host-ts": 3386}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:20f3c2d4-19a5-40e0-8c97-7b3edb28aec3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:20f3c2d4-19a5-40e0-8c97-7b3edb28aec3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fe524544 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fe524544 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fe524544 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3398 (Thu Oct 4 09:43:50 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3398 (Thu Oct 4 09:43:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0e90c296", "local_conf_timestamp": 3398, "host-ts": 3398}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3396 (Thu Oct 4 09:43:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3396 (Thu Oct 4 09:43:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "58cf6a7e", "local_conf_timestamp": 3396, "host-ts": 3396}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:9f48c0ef-ebcc-4a54-80d4-4d3e243de50e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9f48c0ef-ebcc-4a54-80d4-4d3e243de50e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 040a95b8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 040a95b8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 040a95b8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3398 (Thu Oct 4 09:43:50 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3398 (Thu Oct 4 09:43:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0e90c296", "local_conf_timestamp": 3398, "host-ts": 3398}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3396 (Thu Oct 4 09:43:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3396 (Thu Oct 4 09:43:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "58cf6a7e", "local_conf_timestamp": 3396, "host-ts": 3396}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:429cd235-4fe5-4c16-b3b0-11f9e55560ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:429cd235-4fe5-4c16-b3b0-11f9e55560ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 076c265e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 076c265e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 076c265e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3408 (Thu Oct 4 09:44:01 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3409 (Thu Oct 4 09:44:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a95b9e55", "local_conf_timestamp": 3409, "host-ts": 3408}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3406 (Thu Oct 4 09:43:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3406 (Thu Oct 4 09:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f4c67442", "local_conf_timestamp": 3406, "host-ts": 3406}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:2a141c53-21e1-4165-b46e-1cf570715f1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2a141c53-21e1-4165-b46e-1cf570715f1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0b16e942 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0b16e942 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0b16e942 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3408 (Thu Oct 4 09:44:01 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3409 (Thu Oct 4 09:44:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a95b9e55", "local_conf_timestamp": 3409, "host-ts": 3408}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3406 (Thu Oct 4 09:43:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3406 (Thu Oct 4 09:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f4c67442", "local_conf_timestamp": 3406, "host-ts": 3406}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:e486af2a-d1e1-4b09-ae1b-4c295bac8e90:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e486af2a-d1e1-4b09-ae1b-4c295bac8e90:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0ec55f06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0ec55f06 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0ec55f06 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3418 (Thu Oct 4 09:44:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3418 (Thu Oct 4 09:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f09649f", "local_conf_timestamp": 3418, "host-ts": 3418}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3416 (Thu Oct 4 09:44:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3416 (Thu Oct 4 09:44:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f0808b47", "local_conf_timestamp": 3416, "host-ts": 3416}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:f3668d90-a0c3-4f38-b486-1f5efa7f7c88:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f3668d90-a0c3-4f38-b486-1f5efa7f7c88:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1261b772 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1261b772 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1261b772 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3428 (Thu Oct 4 09:44:21 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3429 (Thu Oct 4 09:44:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2bf00aed", "local_conf_timestamp": 3429, "host-ts": 3428}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3426 (Thu Oct 4 09:44:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3426 (Thu Oct 4 09:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fc4b8a48", "local_conf_timestamp": 3426, "host-ts": 3426}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:79c50e1a-ea5f-44e3-a92c-dbdad9dd29e2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:79c50e1a-ea5f-44e3-a92c-dbdad9dd29e2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 15ac24d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 15ac24d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 15ac24d0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3428 (Thu Oct 4 09:44:21 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3429 (Thu Oct 4 09:44:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2bf00aed", "local_conf_timestamp": 3429, "host-ts": 3428}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3426 (Thu Oct 4 09:44:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3426 (Thu Oct 4 09:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fc4b8a48", "local_conf_timestamp": 3426, "host-ts": 3426}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:8ea9732b-ebbf-479e-a5f8-c33e269564f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8ea9732b-ebbf-479e-a5f8-c33e269564f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 18eb5c92 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 18eb5c92 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 18eb5c92 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3438 (Thu Oct 4 09:44:31 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3438 (Thu Oct 4 09:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1da2f027", "local_conf_timestamp": 3438, "host-ts": 3438}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3436 (Thu Oct 4 09:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3436 (Thu Oct 4 09:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f80d754d", "local_conf_timestamp": 3436, "host-ts": 3436}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:f95ac86b-781d-4bc6-8a9a-a1a6224f395d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f95ac86b-781d-4bc6-8a9a-a1a6224f395d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1d18356a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1d18356a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1d18356a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3448 (Thu Oct 4 09:44:41 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3448 (Thu Oct 4 09:44:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "007a81f2", "local_conf_timestamp": 3448, "host-ts": 3448}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3446 (Thu Oct 4 09:44:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3447 (Thu Oct 4 09:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "92dab8c0", "local_conf_timestamp": 3447, "host-ts": 3446}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:b0263ebe-2821-49e8-b0a7-2bfbb224a43b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b0263ebe-2821-49e8-b0a7-2bfbb224a43b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2179aae4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2179aae4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2179aae4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3448 (Thu Oct 4 09:44:41 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3448 (Thu Oct 4 09:44:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "007a81f2", "local_conf_timestamp": 3448, "host-ts": 3448}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3446 (Thu Oct 4 09:44:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3447 (Thu Oct 4 09:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "92dab8c0", "local_conf_timestamp": 3447, "host-ts": 3446}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:456b4cca-60ce-4282-a4ea-a675560048e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:456b4cca-60ce-4282-a4ea-a675560048e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 253bc464 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 253bc464 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 253bc464 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3458 (Thu Oct 4 09:44:50 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3458 (Thu Oct 4 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "56af83fa", "local_conf_timestamp": 3458, "host-ts": 3458}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3456 (Thu Oct 4 09:44:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3456 (Thu Oct 4 09:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e19b7753", "local_conf_timestamp": 3456, "host-ts": 3456}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:c63e8892-0155-4afc-a41d-f35f4eb94cd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c63e8892-0155-4afc-a41d-f35f4eb94cd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 28fdd538 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 28fdd538 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 28fdd538 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3468 (Thu Oct 4 09:45:01 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3468 (Thu Oct 4 09:45:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9551dd1e", "local_conf_timestamp": 3468, "host-ts": 3468}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3466 (Thu Oct 4 09:44:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3466 (Thu Oct 4 09:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed50765c", "local_conf_timestamp": 3466, "host-ts": 3466}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:7297b9f6-3b34-4367-a8f8-f6fdd2e32ded:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7297b9f6-3b34-4367-a8f8-f6fdd2e32ded:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2d8da9ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2d8da9ca on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2d8da9ca on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3468 (Thu Oct 4 09:45:01 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3468 (Thu Oct 4 09:45:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9551dd1e", "local_conf_timestamp": 3468, "host-ts": 3468}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3466 (Thu Oct 4 09:44:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3466 (Thu Oct 4 09:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed50765c", "local_conf_timestamp": 3466, "host-ts": 3466}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:d56805c8-ada4-4e0d-b735-22b02bcd3e91:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d56805c8-ada4-4e0d-b735-22b02bcd3e91:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 32f8f45a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 32f8f45a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 32f8f45a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3478 (Thu Oct 4 09:45:10 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3478 (Thu Oct 4 09:45:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d4041742", "local_conf_timestamp": 3478, "host-ts": 3478}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3476 (Thu Oct 4 09:45:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3476 (Thu Oct 4 09:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e9168959", "local_conf_timestamp": 3476, "host-ts": 3476}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:f339adce-4d16-4509-a178-0f0c89fb9acc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f339adce-4d16-4509-a178-0f0c89fb9acc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 36c35c4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 36c35c4c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 36c35c4c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3488 (Thu Oct 4 09:45:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3489 (Thu Oct 4 09:45:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9e60e22", "local_conf_timestamp": 3489, "host-ts": 3488}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3486 (Thu Oct 4 09:45:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3486 (Thu Oct 4 09:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d6f18c6a", "local_conf_timestamp": 3486, "host-ts": 3486}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:108b90d6-a84d-4089-8f0f-c5f017828dea:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:108b90d6-a84d-4089-8f0f-c5f017828dea:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3b44df52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3b44df52 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3b44df52 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3498 (Thu Oct 4 09:45:31 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3498 (Thu Oct 4 09:45:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "efb4f4e8", "local_conf_timestamp": 3498, "host-ts": 3498}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3496 (Thu Oct 4 09:45:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3496 (Thu Oct 4 09:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d2b7736f", "local_conf_timestamp": 3496, "host-ts": 3496}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:705dfd9b-7751-4274-a837-69b6bc0c3228:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:705dfd9b-7751-4274-a837-69b6bc0c3228:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3f92ff80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/371/ Build Number: 371 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/372/ Build Number: 372 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/373/ Build Number: 373 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.clear_global_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 76, in clear_global_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/374/ Build Number: 374 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:04a050ea-d4cb-4a0f-ba1a-c3dc539047bf:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:04a050ea-d4cb-4a0f-ba1a-c3dc539047bf:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running a99f50c0 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a99f50c0 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command a99f50c0 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Fri Oct 5 17:39:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4199 (Fri Oct 5 17:39:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4be0b692", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Fri Oct 5 17:39:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4206 (Fri Oct 5 17:39:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:c131f42c-3a22-4b98-a714-f342aaa4db4a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c131f42c-3a22-4b98-a714-f342aaa4db4a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aace3ede on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command aace3ede on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:76246446-9c01-4c44-b2ad-3f2a11c1d57c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76246446-9c01-4c44-b2ad-3f2a11c1d57c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running abdd5dfa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command abdd5dfa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command abdd5dfa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Fri Oct 5 17:39:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4208 (Fri Oct 5 17:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5b00040", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Fri Oct 5 17:39:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4206 (Fri Oct 5 17:39:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:007a949e-26b7-436c-b13d-43a8c3ddf722:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:007a949e-26b7-436c-b13d-43a8c3ddf722:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b2f0bdbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b2f0bdbc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b2f0bdbc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Fri Oct 5 17:40:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Fri Oct 5 17:40:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Oct 5 17:40:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Fri Oct 5 17:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:cab156b6-b620-44c4-9c41-78a6615a1b64:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cab156b6-b620-44c4-9c41-78a6615a1b64:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b6184af0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b6184af0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b6184af0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Fri Oct 5 17:40:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Fri Oct 5 17:40:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Oct 5 17:40:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Fri Oct 5 17:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:88de1ae1-b524-4148-8d5a-4a92225f3b79:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:88de1ae1-b524-4148-8d5a-4a92225f3b79:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b9160db4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b9160db4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b9160db4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Fri Oct 5 17:40:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Fri Oct 5 17:40:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Oct 5 17:40:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4236 (Fri Oct 5 17:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1f1cb35f", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false} lago.ssh: DEBUG: start task:74b25891-b194-49eb-bcfc-247e8df7ffc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:74b25891-b194-49eb-bcfc-247e8df7ffc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bbfb3086 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bbfb3086 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bbfb3086 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Fri Oct 5 17:40:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Fri Oct 5 17:40:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Oct 5 17:40:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4236 (Fri Oct 5 17:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1f1cb35f", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false} lago.ssh: DEBUG: start task:879fd23b-2d2f-401e-9efb-21e6bb9ba529:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:879fd23b-2d2f-401e-9efb-21e6bb9ba529:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bedc193c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bedc193c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bedc193c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Fri Oct 5 17:40:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4248 (Fri Oct 5 17:40:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Fri Oct 5 17:40:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Fri Oct 5 17:40:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false} lago.ssh: DEBUG: start task:16cc3b6a-a5fd-4f41-ab60-d5ad3f824718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:16cc3b6a-a5fd-4f41-ab60-d5ad3f824718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c248bc06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c248bc06 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c248bc06 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Fri Oct 5 17:40:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4248 (Fri Oct 5 17:40:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Fri Oct 5 17:40:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Fri Oct 5 17:40:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false} lago.ssh: DEBUG: start task:27804475-899b-4055-825d-d2670ceee4df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:27804475-899b-4055-825d-d2670ceee4df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c52d649e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c52d649e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c52d649e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Fri Oct 5 17:40:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Fri Oct 5 17:40:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Fri Oct 5 17:40:34 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4256 (Fri Oct 5 17:40:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b5ae69cf", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false} lago.ssh: DEBUG: start task:33992555-b8ce-4be9-adc6-71389e3d61f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:33992555-b8ce-4be9-adc6-71389e3d61f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c81d40a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c81d40a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c81d40a2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Fri Oct 5 17:40:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Fri Oct 5 17:40:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Fri Oct 5 17:40:34 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4256 (Fri Oct 5 17:40:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b5ae69cf", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false} lago.ssh: DEBUG: start task:d7d30a9b-b183-417a-ac0f-0408bb03c9ac:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d7d30a9b-b183-417a-ac0f-0408bb03c9ac:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb2121ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cb2121ec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cb2121ec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Fri Oct 5 17:40:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Fri Oct 5 17:40:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cd47174f", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Fri Oct 5 17:40:44 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4266 (Fri Oct 5 17:40:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false} lago.ssh: DEBUG: start task:cd5429d4-427f-4655-b10a-e2ba5cef4d80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cd5429d4-427f-4655-b10a-e2ba5cef4d80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ce2b46f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ce2b46f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ce2b46f6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Fri Oct 5 17:40:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Fri Oct 5 17:40:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cd47174f", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Fri Oct 5 17:40:44 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4266 (Fri Oct 5 17:40:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false} lago.ssh: DEBUG: start task:3f60ff47-0d4d-46ac-aa55-53b1b5e5c9c6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3f60ff47-0d4d-46ac-aa55-53b1b5e5c9c6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d12eb162 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d12eb162 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d12eb162 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Fri Oct 5 17:40:57 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4278 (Fri Oct 5 17:40:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3750ef7d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4276 (Fri Oct 5 17:40:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4276 (Fri Oct 5 17:40:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "bd2397c5", "local_conf_timestamp": 4276, "host-ts": 4276}, "global_maintenance": false} lago.ssh: DEBUG: start task:c5a8f3c8-45b6-4f05-9185-ddb965cf40f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c5a8f3c8-45b6-4f05-9185-ddb965cf40f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d2491524 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d2491524 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d2491524 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Fri Oct 5 17:40:57 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4278 (Fri Oct 5 17:40:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3750ef7d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4276 (Fri Oct 5 17:40:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4276 (Fri Oct 5 17:40:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "bd2397c5", "local_conf_timestamp": 4276, "host-ts": 4276}, "global_maintenance": false} lago.ssh: DEBUG: start task:a388f885-1da3-4d0b-acb0-a12844028e3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a388f885-1da3-4d0b-acb0-a12844028e3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d94c5a7a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d94c5a7a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d94c5a7a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4288 (Fri Oct 5 17:41:06 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4288 (Fri Oct 5 17:41:06 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "9390fe83", "local_conf_timestamp": 4288, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4286 (Fri Oct 5 17:41:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4286 (Fri Oct 5 17:41:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:04a050ea-d4cb-4a0f-ba1a-c3dc539047bf:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:04a050ea-d4cb-4a0f-ba1a-c3dc539047bf:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running a99f50c0 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a99f50c0 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command a99f50c0 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Fri Oct 5 17:39:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4199 (Fri Oct 5 17:39:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4be0b692", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Fri Oct 5 17:39:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4206 (Fri Oct 5 17:39:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c131f42c-3a22-4b98-a714-f342aaa4db4a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c131f42c-3a22-4b98-a714-f342aaa4db4a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aace3ede on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command aace3ede on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:76246446-9c01-4c44-b2ad-3f2a11c1d57c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76246446-9c01-4c44-b2ad-3f2a11c1d57c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running abdd5dfa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command abdd5dfa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command abdd5dfa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Fri Oct 5 17:39:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4208 (Fri Oct 5 17:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5b00040", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Fri Oct 5 17:39:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4206 (Fri Oct 5 17:39:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:007a949e-26b7-436c-b13d-43a8c3ddf722:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:007a949e-26b7-436c-b13d-43a8c3ddf722:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b2f0bdbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b2f0bdbc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b2f0bdbc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Fri Oct 5 17:40:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Fri Oct 5 17:40:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Oct 5 17:40:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Fri Oct 5 17:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cab156b6-b620-44c4-9c41-78a6615a1b64:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cab156b6-b620-44c4-9c41-78a6615a1b64:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b6184af0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b6184af0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b6184af0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Fri Oct 5 17:40:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Fri Oct 5 17:40:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Oct 5 17:40:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Fri Oct 5 17:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:88de1ae1-b524-4148-8d5a-4a92225f3b79:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:88de1ae1-b524-4148-8d5a-4a92225f3b79:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b9160db4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b9160db4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b9160db4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Fri Oct 5 17:40:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Fri Oct 5 17:40:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Oct 5 17:40:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4236 (Fri Oct 5 17:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1f1cb35f", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:74b25891-b194-49eb-bcfc-247e8df7ffc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:74b25891-b194-49eb-bcfc-247e8df7ffc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bbfb3086 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bbfb3086 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bbfb3086 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Fri Oct 5 17:40:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Fri Oct 5 17:40:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Oct 5 17:40:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4236 (Fri Oct 5 17:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1f1cb35f", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:879fd23b-2d2f-401e-9efb-21e6bb9ba529:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:879fd23b-2d2f-401e-9efb-21e6bb9ba529:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bedc193c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bedc193c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bedc193c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Fri Oct 5 17:40:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4248 (Fri Oct 5 17:40:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Fri Oct 5 17:40:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Fri Oct 5 17:40:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:16cc3b6a-a5fd-4f41-ab60-d5ad3f824718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:16cc3b6a-a5fd-4f41-ab60-d5ad3f824718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c248bc06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c248bc06 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c248bc06 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Fri Oct 5 17:40:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4248 (Fri Oct 5 17:40:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Fri Oct 5 17:40:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Fri Oct 5 17:40:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:27804475-899b-4055-825d-d2670ceee4df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:27804475-899b-4055-825d-d2670ceee4df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c52d649e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c52d649e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c52d649e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Fri Oct 5 17:40:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Fri Oct 5 17:40:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Fri Oct 5 17:40:34 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4256 (Fri Oct 5 17:40:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b5ae69cf", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:33992555-b8ce-4be9-adc6-71389e3d61f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:33992555-b8ce-4be9-adc6-71389e3d61f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c81d40a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c81d40a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c81d40a2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Fri Oct 5 17:40:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Fri Oct 5 17:40:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Fri Oct 5 17:40:34 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4256 (Fri Oct 5 17:40:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b5ae69cf", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d7d30a9b-b183-417a-ac0f-0408bb03c9ac:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d7d30a9b-b183-417a-ac0f-0408bb03c9ac:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb2121ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cb2121ec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cb2121ec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Fri Oct 5 17:40:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Fri Oct 5 17:40:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cd47174f", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Fri Oct 5 17:40:44 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4266 (Fri Oct 5 17:40:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cd5429d4-427f-4655-b10a-e2ba5cef4d80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cd5429d4-427f-4655-b10a-e2ba5cef4d80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ce2b46f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ce2b46f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ce2b46f6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Fri Oct 5 17:40:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Fri Oct 5 17:40:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cd47174f", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Fri Oct 5 17:40:44 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4266 (Fri Oct 5 17:40:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3f60ff47-0d4d-46ac-aa55-53b1b5e5c9c6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3f60ff47-0d4d-46ac-aa55-53b1b5e5c9c6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d12eb162 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d12eb162 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d12eb162 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Fri Oct 5 17:40:57 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4278 (Fri Oct 5 17:40:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3750ef7d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4276 (Fri Oct 5 17:40:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4276 (Fri Oct 5 17:40:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "bd2397c5", "local_conf_timestamp": 4276, "host-ts": 4276}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c5a8f3c8-45b6-4f05-9185-ddb965cf40f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c5a8f3c8-45b6-4f05-9185-ddb965cf40f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d2491524 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d2491524 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d2491524 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Fri Oct 5 17:40:57 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4278 (Fri Oct 5 17:40:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3750ef7d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4276 (Fri Oct 5 17:40:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4276 (Fri Oct 5 17:40:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "bd2397c5", "local_conf_timestamp": 4276, "host-ts": 4276}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a388f885-1da3-4d0b-acb0-a12844028e3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a388f885-1da3-4d0b-acb0-a12844028e3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d94c5a7a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d94c5a7a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d94c5a7a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4288 (Fri Oct 5 17:41:06 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4288 (Fri Oct 5 17:41:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "9390fe83", "local_conf_timestamp": 4288, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4286 (Fri Oct 5 17:41:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4286 (Fri Oct 5 17:41:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/375/ Build Number: 375 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:d69ebaa9-1c23-4f3e-ac56-0e421e7c5f47:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:d69ebaa9-1c23-4f3e-ac56-0e421e7c5f47:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running f8f13744 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 119, in local_maintenance he_index, host = _find_host_running_he_vm(hosts) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 80, in _find_host_running_he_vm status = _get_he_status(hosts[0]) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 92, in _get_he_status ret = host.ssh(["hosted-engine", "--vm-status", "--json"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 61, in ssh channel.exec_command(joined_command) File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 60, in _check return func(self, *args, **kwds) File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 234, in exec_command self._wait_for_event() File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 1154, in _wait_for_event self.event.wait() File "/usr/lib64/python2.7/threading.py", line 622, in wait self.__cond.wait(timeout, balancing) File "/usr/lib64/python2.7/threading.py", line 339, in wait waiter.acquire() File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:d69ebaa9-1c23-4f3e-ac56-0e421e7c5f47:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:d69ebaa9-1c23-4f3e-ac56-0e421e7c5f47:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running f8f13744 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/376/ Build Number: 376 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.restart_he_vm Error Message: could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-st -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:c44edc62-5de4-46f8-a4da-91d5e60d519b:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:c44edc62-5de4-46f8-a4da-91d5e60d519b:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 1858959c on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1858959c on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 1858959c on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3243 (Sat Oct 6 09:27:43 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3243 (Sat Oct 6 09:27:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3a5d8fcb", "local_conf_timestamp": 3243, "host-ts": 3243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3244 (Sat Oct 6 09:27:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3244 (Sat Oct 6 09:27:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ff08047a", "local_conf_timestamp": 3244, "host-ts": 3244}, "global_maintenance": true} root: INFO: * Shutting down HE VM on host: lago-he-node-ng-suite-4-2-host-0 lago.ssh: DEBUG: start task:8bec795e-0a02-4d83-8919-c388a99e8798:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8bec795e-0a02-4d83-8919-c388a99e8798:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1963e130 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-shutdown lago.ssh: DEBUG: Command 1963e130 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Command succeeded root: INFO: * Waiting for VM to be down... lago.ssh: DEBUG: start task:1a13a370-162f-4fe7-bc9c-e0c25121d6a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1a13a370-162f-4fe7-bc9c-e0c25121d6a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1bd51f4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1bd51f4c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1bd51f4c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3253 (Sat Oct 6 09:27:53 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3253 (Sat Oct 6 09:27:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e63acff4", "local_conf_timestamp": 3253, "host-ts": 3253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3244 (Sat Oct 6 09:27:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3244 (Sat Oct 6 09:27:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ff08047a", "local_conf_timestamp": 3244, "host-ts": 3244}, "global_maintenance": true} lago.ssh: DEBUG: start task:5765e57d-473f-433f-98e4-03da4b175b7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5765e57d-473f-433f-98e4-03da4b175b7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 237c9734 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 237c9734 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 237c9734 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3263 (Sat Oct 6 09:28:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3264 (Sat Oct 6 09:28:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "202ccf78", "local_conf_timestamp": 3264, "host-ts": 3263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3254 (Sat Oct 6 09:27:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3254 (Sat Oct 6 09:27:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fb4efb7f", "local_conf_timestamp": 3254, "host-ts": 3254}, "global_maintenance": true} lago.ssh: DEBUG: start task:0e1588f5-b1e6-4a80-b9d7-597f9ff9d718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0e1588f5-b1e6-4a80-b9d7-597f9ff9d718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 265cd4be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 265cd4be on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 265cd4be on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3273 (Sat Oct 6 09:28:13 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3273 (Sat Oct 6 09:28:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "486617f5", "local_conf_timestamp": 3273, "host-ts": 3273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3264 (Sat Oct 6 09:28:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3264 (Sat Oct 6 09:28:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f785fa70", "local_conf_timestamp": 3264, "host-ts": 3264}, "global_maintenance": true} root: INFO: * VM is down. root: INFO: * Stopping services... lago.ssh: DEBUG: start task:0db6b7b8-6e36-40bf-9739-f392e79e2b3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0db6b7b8-6e36-40bf-9739-f392e79e2b3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 278add22 on lago-he-node-ng-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command 278add22 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Starting services... lago.ssh: DEBUG: start task:af03fd03-5b53-4480-89f6-5bb36ee4aa36:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:af03fd03-5b53-4480-89f6-5bb36ee4aa36:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3124aa5c on lago-he-node-ng-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command 3124aa5c on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for agent to be ready... lago.ssh: DEBUG: start task:f679ce56-0d10-4f46-b100-a1e336d7d54f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f679ce56-0d10-4f46-b100-a1e336d7d54f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3499986e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 3499986e on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 3499986e on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:bedcd442-f9c9-41a2-88fb-dd2cabbac11c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bedcd442-f9c9-41a2-88fb-dd2cabbac11c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3b9471a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 3b9471a2 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 3b9471a2 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:7417fd76-78ba-41b7-87b0-6083d59e95da:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7417fd76-78ba-41b7-87b0-6083d59e95da:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3e43e9a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 3e43e9a0 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 3e43e9a0 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:6b7af916-fc70-4770-ba37-a0d8c8df1a8c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6b7af916-fc70-4770-ba37-a0d8c8df1a8c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 40f50a80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 40f50a80 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 40f50a80 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:e39ae262-0351-4a14-8600-81ce96de8ad8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e39ae262-0351-4a14-8600-81ce96de8ad8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 43aff550 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 43aff550 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 43aff550 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:b8b09ab3-a843-4591-880a-4e1eaa4e4948:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b8b09ab3-a843-4591-880a-4e1eaa4e4948:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 46571770 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 46571770 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 46571770 on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:9ab6204c-2b64-4e80-86cf-19c1a4b74af8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9ab6204c-2b64-4e80-86cf-19c1a4b74af8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4905ca5c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 4905ca5c on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 4905ca5c on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:9b25c69b-0bf2-4e55-b4f7-e2dda7067c49:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9b25c69b-0bf2-4e55-b4f7-e2dda7067c49:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4bb6bfcc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 4bb6bfcc on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 4bb6bfcc on lago-he-node-ng-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:9feb9a9f-a76c-4e3a-8ece-7c6f5c81563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9feb9a9f-a76c-4e3a-8ece-7c6f5c81563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4e74bfb6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 4e74bfb6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4e74bfb6 on lago-he-node-ng-suite-4-2-host-0 output: !! Cluster is in GLOBAL MAINTENANCE mode !! --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : lago-he-node-ng-suite-4-2-host-0 Host ID : 1 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"} Score : 0 stopped : False Local maintenance : False crc32 : c3a8eba9 local_conf_timestamp : 3337 Host timestamp : 3337 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=3337 (Sat Oct 6 09:29:17 2018) host-id=1 score=0 vm_conf_refresh_time=3337 (Sat Oct 6 09:29:18 2018) conf_on_shared_storage=True maintenance=False state=ReinitializeFSM stopped=False --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : False Hostname : lago-he-node-ng-suite-4-2-host-1 Host ID : 2 Engine status : unknown stale-data Score : 3400 stopped : False Local maintenance : False crc32 : 92b64b11 local_conf_timestamp : 3334 Host timestamp : 3334 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=3334 (Sat Oct 6 09:29:14 2018) host-id=2 score=3400 vm_conf_refresh_time=3334 (Sat Oct 6 09:29:14 2018) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False !! Cluster is in GLOBAL MAINTENANCE mode !! root: INFO: * Agent is ready. root: INFO: * Starting VM... lago.ssh: DEBUG: start task:4e7794c3-69f8-4438-afda-e7fcc7c6b284:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4e7794c3-69f8-4438-afda-e7fcc7c6b284:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4f87dbea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-start lago.ssh: DEBUG: Command 4f87dbea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4f87dbea on lago-he-node-ng-suite-4-2-host-0 output: VM exists and is down, cleaning up and restarting root: INFO: * Command succeeded root: INFO: * Waiting for VM to be UP... lago.ssh: DEBUG: start task:c226a693-a5bf-42ae-8336-2efcfe1b79df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c226a693-a5bf-42ae-8336-2efcfe1b79df:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5179238c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5179238c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5179238c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3337 (Sat Oct 6 09:29:17 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=3337 (Sat Oct 6 09:29:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "c3a8eba9", "local_conf_timestamp": 3337, "host-ts": 3337}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3334 (Sat Oct 6 09:29:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3334 (Sat Oct 6 09:29:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "92b64b11", "local_conf_timestamp": 3334, "host-ts": 3334}, "global_maintenance": true} lago.ssh: DEBUG: start task:f43c6d74-777f-4ce0-9645-45bf215705b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f43c6d74-777f-4ce0-9645-45bf215705b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5889f340 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5889f340 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5889f340 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3357 (Sat Oct 6 09:29:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3357 (Sat Oct 6 09:29:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "efdf572f", "local_conf_timestamp": 3357, "host-ts": 3357}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3354 (Sat Oct 6 09:29:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3354 (Sat Oct 6 09:29:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8b20490f", "local_conf_timestamp": 3354, "host-ts": 3354}, "global_maintenance": true} root: INFO: * VM is UP. root: INFO: * Waiting for engine to start... lago.ssh: DEBUG: start task:e2fdfced-2ef7-4c69-9d88-6f4a824a3ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e2fdfced-2ef7-4c69-9d88-6f4a824a3ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 599c03d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 599c03d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 599c03d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3357 (Sat Oct 6 09:29:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3357 (Sat Oct 6 09:29:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "efdf572f", "local_conf_timestamp": 3357, "host-ts": 3357}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3354 (Sat Oct 6 09:29:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3354 (Sat Oct 6 09:29:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8b20490f", "local_conf_timestamp": 3354, "host-ts": 3354}, "global_maintenance": true} lago.ssh: DEBUG: start task:b6e1e8d4-cd09-4e6e-bb01-dd90e6bf1f0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b6e1e8d4-cd09-4e6e-bb01-dd90e6bf1f0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 60f8d9ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 60f8d9ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 60f8d9ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3367 (Sat Oct 6 09:29:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3367 (Sat Oct 6 09:29:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af2b10ab", "local_conf_timestamp": 3367, "host-ts": 3367}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3364 (Sat Oct 6 09:29:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3364 (Sat Oct 6 09:29:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "87eb4800", "local_conf_timestamp": 3364, "host-ts": 3364}, "global_maintenance": true} lago.ssh: DEBUG: start task:2ba187ca-4d15-4830-b47f-31fd2039a6c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2ba187ca-4d15-4830-b47f-31fd2039a6c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 641750d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 641750d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 641750d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3377 (Sat Oct 6 09:29:58 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3378 (Sat Oct 6 09:29:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "00383046", "local_conf_timestamp": 3378, "host-ts": 3377}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3374 (Sat Oct 6 09:29:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3374 (Sat Oct 6 09:29:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83adb705", "local_conf_timestamp": 3374, "host-ts": 3374}, "global_maintenance": true} lago.ssh: DEBUG: start task:9a7facd3-b79b-4d7d-8d2c-b7fbdab7339b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9a7facd3-b79b-4d7d-8d2c-b7fbdab7339b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6737a216 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6737a216 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6737a216 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3377 (Sat Oct 6 09:29:58 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3378 (Sat Oct 6 09:29:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "00383046", "local_conf_timestamp": 3378, "host-ts": 3377}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3374 (Sat Oct 6 09:29:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3374 (Sat Oct 6 09:29:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83adb705", "local_conf_timestamp": 3374, "host-ts": 3374}, "global_maintenance": true} lago.ssh: DEBUG: start task:d0a5cf4e-015e-462c-a666-eb7ed63aa597:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d0a5cf4e-015e-462c-a666-eb7ed63aa597:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6a5cf630 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6a5cf630 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6a5cf630 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3387 (Sat Oct 6 09:30:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3387 (Sat Oct 6 09:30:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "08d37202", "local_conf_timestamp": 3387, "host-ts": 3387}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3384 (Sat Oct 6 09:30:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3384 (Sat Oct 6 09:30:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc4ab236", "local_conf_timestamp": 3384, "host-ts": 3384}, "global_maintenance": true} lago.ssh: DEBUG: start task:8f5d407a-b6f9-41ac-994e-f35daabeea00:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8f5d407a-b6f9-41ac-994e-f35daabeea00:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6d99d3ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6d99d3ea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6d99d3ea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3387 (Sat Oct 6 09:30:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3387 (Sat Oct 6 09:30:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "08d37202", "local_conf_timestamp": 3387, "host-ts": 3387}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3384 (Sat Oct 6 09:30:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3384 (Sat Oct 6 09:30:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc4ab236", "local_conf_timestamp": 3384, "host-ts": 3384}, "global_maintenance": true} lago.ssh: DEBUG: start task:84bdb8e5-5053-42df-9e0c-16602f1c8b77:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:84bdb8e5-5053-42df-9e0c-16602f1c8b77:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71082d6a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71082d6a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71082d6a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3397 (Sat Oct 6 09:30:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3398 (Sat Oct 6 09:30:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "115c818c", "local_conf_timestamp": 3398, "host-ts": 3397}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3394 (Sat Oct 6 09:30:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3394 (Sat Oct 6 09:30:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b80c4d33", "local_conf_timestamp": 3394, "host-ts": 3394}, "global_maintenance": true} lago.ssh: DEBUG: start task:bc79cad1-a004-4743-888b-0e702b36a275:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bc79cad1-a004-4743-888b-0e702b36a275:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 750dddec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 750dddec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 750dddec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3407 (Sat Oct 6 09:30:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3407 (Sat Oct 6 09:30:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "512ff048", "local_conf_timestamp": 3407, "host-ts": 3407}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3404 (Sat Oct 6 09:30:23 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3404 (Sat Oct 6 09:30:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1405530f", "local_conf_timestamp": 3404, "host-ts": 3404}, "global_maintenance": true} lago.ssh: DEBUG: start task:63bb20ac-5a36-406e-be57-1075c035119c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:63bb20ac-5a36-406e-be57-1075c035119c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 794fe4f4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 794fe4f4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 794fe4f4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3407 (Sat Oct 6 09:30:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3407 (Sat Oct 6 09:30:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "512ff048", "local_conf_timestamp": 3407, "host-ts": 3407}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3404 (Sat Oct 6 09:30:23 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3404 (Sat Oct 6 09:30:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1405530f", "local_conf_timestamp": 3404, "host-ts": 3404}, "global_maintenance": true} lago.ssh: DEBUG: start task:c1557713-869d-4318-a036-fbdccd32c9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1557713-869d-4318-a036-fbdccd32c9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7c750740 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7c750740 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7c750740 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3417 (Sat Oct 6 09:30:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3417 (Sat Oct 6 09:30:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "107a3a14", "local_conf_timestamp": 3417, "host-ts": 3417}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3414 (Sat Oct 6 09:30:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3414 (Sat Oct 6 09:30:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1043ac0a", "local_conf_timestamp": 3414, "host-ts": 3414}, "global_maintenance": true} lago.ssh: DEBUG: start task:d339c7c0-0ef6-47d0-b369-e2bc29b628bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d339c7c0-0ef6-47d0-b369-e2bc29b628bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7f940cdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7f940cdc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7f940cdc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3417 (Sat Oct 6 09:30:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3417 (Sat Oct 6 09:30:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "107a3a14", "local_conf_timestamp": 3417, "host-ts": 3417}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3414 (Sat Oct 6 09:30:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3414 (Sat Oct 6 09:30:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1043ac0a", "local_conf_timestamp": 3414, "host-ts": 3414}, "global_maintenance": true} lago.ssh: DEBUG: start task:4a03aa6b-296c-40ba-b112-31133d8b32eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4a03aa6b-296c-40ba-b112-31133d8b32eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 82ea743e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 82ea743e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 82ea743e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3427 (Sat Oct 6 09:30:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3427 (Sat Oct 6 09:30:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d38464f0", "local_conf_timestamp": 3427, "host-ts": 3427}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3424 (Sat Oct 6 09:30:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3424 (Sat Oct 6 09:30:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c88ad05", "local_conf_timestamp": 3424, "host-ts": 3424}, "global_maintenance": true} lago.ssh: DEBUG: start task:4a7239e6-0641-4061-a9e1-705ac71fbdc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4a7239e6-0641-4061-a9e1-705ac71fbdc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8659fdc4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8659fdc4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8659fdc4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3427 (Sat Oct 6 09:30:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3427 (Sat Oct 6 09:30:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d38464f0", "local_conf_timestamp": 3427, "host-ts": 3427}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3424 (Sat Oct 6 09:30:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3424 (Sat Oct 6 09:30:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c88ad05", "local_conf_timestamp": 3424, "host-ts": 3424}, "global_maintenance": true} lago.ssh: DEBUG: start task:675b186f-f933-4c72-a848-464ceed95677:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:675b186f-f933-4c72-a848-464ceed95677:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 89779624 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 89779624 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 89779624 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3437 (Sat Oct 6 09:30:58 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3438 (Sat Oct 6 09:30:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "026eb33d", "local_conf_timestamp": 3438, "host-ts": 3437}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3434 (Sat Oct 6 09:30:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3434 (Sat Oct 6 09:30:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18ce5200", "local_conf_timestamp": 3434, "host-ts": 3434}, "global_maintenance": true} lago.ssh: DEBUG: start task:500ffb6c-4171-49ac-8f11-636dc32057cd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:500ffb6c-4171-49ac-8f11-636dc32057cd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8cd7ec92 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8cd7ec92 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8cd7ec92 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3437 (Sat Oct 6 09:30:58 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3438 (Sat Oct 6 09:30:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "026eb33d", "local_conf_timestamp": 3438, "host-ts": 3437}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3434 (Sat Oct 6 09:30:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3434 (Sat Oct 6 09:30:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18ce5200", "local_conf_timestamp": 3434, "host-ts": 3434}, "global_maintenance": true} lago.ssh: DEBUG: start task:7aeb35e4-0e21-425b-9e0b-db3eb92cfb09:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7aeb35e4-0e21-425b-9e0b-db3eb92cfb09:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 903e89d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 903e89d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 903e89d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3447 (Sat Oct 6 09:31:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3447 (Sat Oct 6 09:31:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8f09df79", "local_conf_timestamp": 3447, "host-ts": 3447}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3444 (Sat Oct 6 09:31:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3445 (Sat Oct 6 09:31:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "72199f8d", "local_conf_timestamp": 3445, "host-ts": 3444}, "global_maintenance": true} lago.ssh: DEBUG: start task:0fe988ff-f225-4c70-b20e-d4941d8bda17:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0fe988ff-f225-4c70-b20e-d4941d8bda17:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 945ceb20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 945ceb20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 945ceb20 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3457 (Sat Oct 6 09:31:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3457 (Sat Oct 6 09:31:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ce5c1525", "local_conf_timestamp": 3457, "host-ts": 3457}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3454 (Sat Oct 6 09:31:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3454 (Sat Oct 6 09:31:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0158501e", "local_conf_timestamp": 3454, "host-ts": 3454}, "global_maintenance": true} lago.ssh: DEBUG: start task:b5c9a918-654d-4ce7-be1e-286bfc26cf34:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b5c9a918-654d-4ce7-be1e-286bfc26cf34:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 97c6f864 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 97c6f864 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 97c6f864 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3457 (Sat Oct 6 09:31:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3457 (Sat Oct 6 09:31:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ce5c1525", "local_conf_timestamp": 3457, "host-ts": 3457}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3454 (Sat Oct 6 09:31:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3454 (Sat Oct 6 09:31:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0158501e", "local_conf_timestamp": 3454, "host-ts": 3454}, "global_maintenance": true} lago.ssh: DEBUG: start task:49e56adb-d202-4589-8416-8164b2821d27:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:49e56adb-d202-4589-8416-8164b2821d27:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9bc30e08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9bc30e08 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9bc30e08 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3467 (Sat Oct 6 09:31:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3467 (Sat Oct 6 09:31:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1a228395", "local_conf_timestamp": 3467, "host-ts": 3467}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3464 (Sat Oct 6 09:31:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3464 (Sat Oct 6 09:31:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d935111", "local_conf_timestamp": 3464, "host-ts": 3464}, "global_maintenance": true} lago.ssh: DEBUG: start task:df31cbc7-427e-4dee-a82a-688d53a69dc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:df31cbc7-427e-4dee-a82a-688d53a69dc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a04dac08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a04dac08 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a04dac08 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3477 (Sat Oct 6 09:31:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3478 (Sat Oct 6 09:31:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cbc85458", "local_conf_timestamp": 3478, "host-ts": 3477}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3474 (Sat Oct 6 09:31:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3474 (Sat Oct 6 09:31:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "09d5ae14", "local_conf_timestamp": 3474, "host-ts": 3474}, "global_maintenance": true} lago.ssh: DEBUG: start task:3153a30a-4831-4644-9f65-e62022c548dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3153a30a-4831-4644-9f65-e62022c548dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a4da2986 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a4da2986 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a4da2986 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3487 (Sat Oct 6 09:31:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3487 (Sat Oct 6 09:31:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2192603f", "local_conf_timestamp": 3487, "host-ts": 3487}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3484 (Sat Oct 6 09:31:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3484 (Sat Oct 6 09:31:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3632ab27", "local_conf_timestamp": 3484, "host-ts": 3484}, "global_maintenance": true} lago.ssh: DEBUG: start task:170e0571-d024-4b7f-b125-7679ee0ba1a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:170e0571-d024-4b7f-b125-7679ee0ba1a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a9d7f242 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a9d7f242 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a9d7f242 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3497 (Sat Oct 6 09:31:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3497 (Sat Oct 6 09:31:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "60c7aa63", "local_conf_timestamp": 3497, "host-ts": 3497}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3494 (Sat Oct 6 09:31:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3494 (Sat Oct 6 09:31:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "32745422", "local_conf_timestamp": 3494, "host-ts": 3494}, "global_maintenance": true} lago.ssh: DEBUG: start task:2ef1ee03-ac81-4271-869a-1cd1a44c9d7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2ef1ee03-ac81-4271-869a-1cd1a44c9d7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b008e694 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b008e694 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b008e694 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "646be17f", "local_conf_timestamp": 3504, "host-ts": 3504}, "global_maintenance": true} lago.ssh: DEBUG: start task:19008521-0503-40ee-8f84-3f1e3c125d34:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:19008521-0503-40ee-8f84-3f1e3c125d34:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b5616ada on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b5616ada on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b5616ada on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-st ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f368539e410> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda> for k, v in _get_he_status(host).items() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 128, in _get_he_status raise RuntimeError('could not parse JSON: %s' % ret.out) RuntimeError: could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-st --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 53, in restart_he_vm _wait_for_engine_health(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 185, in _wait_for_engine_health testlib.assert_true_within_long(lambda: any( File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda> for k, v in _get_he_status(host).items() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 128, in _get_he_status raise RuntimeError('could not parse JSON: %s' % ret.out) 'could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-st\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:c44edc62-5de4-46f8-a4da-91d5e60d519b:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:c44edc62-5de4-46f8-a4da-91d5e60d519b:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 1858959c on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1858959c on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 1858959c on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3243 (Sat Oct 6 09:27:43 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3243 (Sat Oct 6 09:27:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3a5d8fcb", "local_conf_timestamp": 3243, "host-ts": 3243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3244 (Sat Oct 6 09:27:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3244 (Sat Oct 6 09:27:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ff08047a", "local_conf_timestamp": 3244, "host-ts": 3244}, "global_maintenance": true}\n\nroot: INFO: * Shutting down HE VM on host: lago-he-node-ng-suite-4-2-host-0\nlago.ssh: DEBUG: start task:8bec795e-0a02-4d83-8919-c388a99e8798:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8bec795e-0a02-4d83-8919-c388a99e8798:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1963e130 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-shutdown\nlago.ssh: DEBUG: Command 1963e130 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be down...\nlago.ssh: DEBUG: start task:1a13a370-162f-4fe7-bc9c-e0c25121d6a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1a13a370-162f-4fe7-bc9c-e0c25121d6a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1bd51f4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1bd51f4c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1bd51f4c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3253 (Sat Oct 6 09:27:53 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3253 (Sat Oct 6 09:27:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e63acff4", "local_conf_timestamp": 3253, "host-ts": 3253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3244 (Sat Oct 6 09:27:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3244 (Sat Oct 6 09:27:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ff08047a", "local_conf_timestamp": 3244, "host-ts": 3244}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:5765e57d-473f-433f-98e4-03da4b175b7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5765e57d-473f-433f-98e4-03da4b175b7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 237c9734 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 237c9734 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 237c9734 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3263 (Sat Oct 6 09:28:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3264 (Sat Oct 6 09:28:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "202ccf78", "local_conf_timestamp": 3264, "host-ts": 3263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3254 (Sat Oct 6 09:27:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3254 (Sat Oct 6 09:27:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fb4efb7f", "local_conf_timestamp": 3254, "host-ts": 3254}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:0e1588f5-b1e6-4a80-b9d7-597f9ff9d718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0e1588f5-b1e6-4a80-b9d7-597f9ff9d718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 265cd4be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 265cd4be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 265cd4be on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3273 (Sat Oct 6 09:28:13 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3273 (Sat Oct 6 09:28:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "486617f5", "local_conf_timestamp": 3273, "host-ts": 3273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3264 (Sat Oct 6 09:28:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3264 (Sat Oct 6 09:28:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f785fa70", "local_conf_timestamp": 3264, "host-ts": 3264}, "global_maintenance": true}\n\nroot: INFO: * VM is down.\nroot: INFO: * Stopping services...\nlago.ssh: DEBUG: start task:0db6b7b8-6e36-40bf-9739-f392e79e2b3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0db6b7b8-6e36-40bf-9739-f392e79e2b3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 278add22 on lago-he-node-ng-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command 278add22 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Starting services...\nlago.ssh: DEBUG: start task:af03fd03-5b53-4480-89f6-5bb36ee4aa36:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:af03fd03-5b53-4480-89f6-5bb36ee4aa36:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3124aa5c on lago-he-node-ng-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command 3124aa5c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for agent to be ready...\nlago.ssh: DEBUG: start task:f679ce56-0d10-4f46-b100-a1e336d7d54f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f679ce56-0d10-4f46-b100-a1e336d7d54f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3499986e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 3499986e on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 3499986e on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:bedcd442-f9c9-41a2-88fb-dd2cabbac11c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bedcd442-f9c9-41a2-88fb-dd2cabbac11c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3b9471a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 3b9471a2 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 3b9471a2 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:7417fd76-78ba-41b7-87b0-6083d59e95da:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7417fd76-78ba-41b7-87b0-6083d59e95da:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3e43e9a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 3e43e9a0 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 3e43e9a0 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:6b7af916-fc70-4770-ba37-a0d8c8df1a8c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6b7af916-fc70-4770-ba37-a0d8c8df1a8c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 40f50a80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 40f50a80 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 40f50a80 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:e39ae262-0351-4a14-8600-81ce96de8ad8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e39ae262-0351-4a14-8600-81ce96de8ad8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 43aff550 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 43aff550 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 43aff550 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:b8b09ab3-a843-4591-880a-4e1eaa4e4948:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b8b09ab3-a843-4591-880a-4e1eaa4e4948:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 46571770 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 46571770 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 46571770 on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:9ab6204c-2b64-4e80-86cf-19c1a4b74af8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9ab6204c-2b64-4e80-86cf-19c1a4b74af8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4905ca5c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 4905ca5c on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 4905ca5c on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:9b25c69b-0bf2-4e55-b4f7-e2dda7067c49:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9b25c69b-0bf2-4e55-b4f7-e2dda7067c49:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4bb6bfcc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 4bb6bfcc on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 4bb6bfcc on lago-he-node-ng-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:9feb9a9f-a76c-4e3a-8ece-7c6f5c81563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9feb9a9f-a76c-4e3a-8ece-7c6f5c81563d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4e74bfb6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 4e74bfb6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4e74bfb6 on lago-he-node-ng-suite-4-2-host-0 output:\n \n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\n\n--== Host 1 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date : True\nHostname : lago-he-node-ng-suite-4-2-host-0\nHost ID : 1\nEngine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}\nScore : 0\nstopped : False\nLocal maintenance : False\ncrc32 : c3a8eba9\nlocal_conf_timestamp : 3337\nHost timestamp : 3337\nExtra metadata (valid at timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=3337 (Sat Oct 6 09:29:17 2018)\n\thost-id=1\n\tscore=0\n\tvm_conf_refresh_time=3337 (Sat Oct 6 09:29:18 2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=ReinitializeFSM\n\tstopped=False\n\n\n--== Host 2 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date : False\nHostname : lago-he-node-ng-suite-4-2-host-1\nHost ID : 2\nEngine status : unknown stale-data\nScore : 3400\nstopped : False\nLocal maintenance : False\ncrc32 : 92b64b11\nlocal_conf_timestamp : 3334\nHost timestamp : 3334\nExtra metadata (valid at timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=3334 (Sat Oct 6 09:29:14 2018)\n\thost-id=2\n\tscore=3400\n\tvm_conf_refresh_time=3334 (Sat Oct 6 09:29:14 2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=GlobalMaintenance\n\tstopped=False\n\n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\nroot: INFO: * Agent is ready.\nroot: INFO: * Starting VM...\nlago.ssh: DEBUG: start task:4e7794c3-69f8-4438-afda-e7fcc7c6b284:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4e7794c3-69f8-4438-afda-e7fcc7c6b284:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4f87dbea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-start\nlago.ssh: DEBUG: Command 4f87dbea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4f87dbea on lago-he-node-ng-suite-4-2-host-0 output:\n VM exists and is down, cleaning up and restarting\n\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be UP...\nlago.ssh: DEBUG: start task:c226a693-a5bf-42ae-8336-2efcfe1b79df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c226a693-a5bf-42ae-8336-2efcfe1b79df:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5179238c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5179238c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5179238c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3337 (Sat Oct 6 09:29:17 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=3337 (Sat Oct 6 09:29:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "c3a8eba9", "local_conf_timestamp": 3337, "host-ts": 3337}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3334 (Sat Oct 6 09:29:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3334 (Sat Oct 6 09:29:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "92b64b11", "local_conf_timestamp": 3334, "host-ts": 3334}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:f43c6d74-777f-4ce0-9645-45bf215705b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f43c6d74-777f-4ce0-9645-45bf215705b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5889f340 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5889f340 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5889f340 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3357 (Sat Oct 6 09:29:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3357 (Sat Oct 6 09:29:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "efdf572f", "local_conf_timestamp": 3357, "host-ts": 3357}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3354 (Sat Oct 6 09:29:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3354 (Sat Oct 6 09:29:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8b20490f", "local_conf_timestamp": 3354, "host-ts": 3354}, "global_maintenance": true}\n\nroot: INFO: * VM is UP.\nroot: INFO: * Waiting for engine to start...\nlago.ssh: DEBUG: start task:e2fdfced-2ef7-4c69-9d88-6f4a824a3ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e2fdfced-2ef7-4c69-9d88-6f4a824a3ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 599c03d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 599c03d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 599c03d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3357 (Sat Oct 6 09:29:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3357 (Sat Oct 6 09:29:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "efdf572f", "local_conf_timestamp": 3357, "host-ts": 3357}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3354 (Sat Oct 6 09:29:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3354 (Sat Oct 6 09:29:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8b20490f", "local_conf_timestamp": 3354, "host-ts": 3354}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:b6e1e8d4-cd09-4e6e-bb01-dd90e6bf1f0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b6e1e8d4-cd09-4e6e-bb01-dd90e6bf1f0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 60f8d9ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 60f8d9ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 60f8d9ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3367 (Sat Oct 6 09:29:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3367 (Sat Oct 6 09:29:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af2b10ab", "local_conf_timestamp": 3367, "host-ts": 3367}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3364 (Sat Oct 6 09:29:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3364 (Sat Oct 6 09:29:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "87eb4800", "local_conf_timestamp": 3364, "host-ts": 3364}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:2ba187ca-4d15-4830-b47f-31fd2039a6c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2ba187ca-4d15-4830-b47f-31fd2039a6c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 641750d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 641750d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 641750d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3377 (Sat Oct 6 09:29:58 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3378 (Sat Oct 6 09:29:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "00383046", "local_conf_timestamp": 3378, "host-ts": 3377}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3374 (Sat Oct 6 09:29:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3374 (Sat Oct 6 09:29:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83adb705", "local_conf_timestamp": 3374, "host-ts": 3374}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:9a7facd3-b79b-4d7d-8d2c-b7fbdab7339b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9a7facd3-b79b-4d7d-8d2c-b7fbdab7339b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6737a216 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6737a216 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6737a216 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3377 (Sat Oct 6 09:29:58 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3378 (Sat Oct 6 09:29:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "00383046", "local_conf_timestamp": 3378, "host-ts": 3377}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3374 (Sat Oct 6 09:29:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3374 (Sat Oct 6 09:29:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83adb705", "local_conf_timestamp": 3374, "host-ts": 3374}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:d0a5cf4e-015e-462c-a666-eb7ed63aa597:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d0a5cf4e-015e-462c-a666-eb7ed63aa597:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6a5cf630 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6a5cf630 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6a5cf630 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3387 (Sat Oct 6 09:30:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3387 (Sat Oct 6 09:30:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "08d37202", "local_conf_timestamp": 3387, "host-ts": 3387}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3384 (Sat Oct 6 09:30:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3384 (Sat Oct 6 09:30:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc4ab236", "local_conf_timestamp": 3384, "host-ts": 3384}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:8f5d407a-b6f9-41ac-994e-f35daabeea00:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8f5d407a-b6f9-41ac-994e-f35daabeea00:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6d99d3ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6d99d3ea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6d99d3ea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3387 (Sat Oct 6 09:30:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3387 (Sat Oct 6 09:30:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "08d37202", "local_conf_timestamp": 3387, "host-ts": 3387}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3384 (Sat Oct 6 09:30:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3384 (Sat Oct 6 09:30:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc4ab236", "local_conf_timestamp": 3384, "host-ts": 3384}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:84bdb8e5-5053-42df-9e0c-16602f1c8b77:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:84bdb8e5-5053-42df-9e0c-16602f1c8b77:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71082d6a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71082d6a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71082d6a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3397 (Sat Oct 6 09:30:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3398 (Sat Oct 6 09:30:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "115c818c", "local_conf_timestamp": 3398, "host-ts": 3397}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3394 (Sat Oct 6 09:30:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3394 (Sat Oct 6 09:30:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b80c4d33", "local_conf_timestamp": 3394, "host-ts": 3394}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:bc79cad1-a004-4743-888b-0e702b36a275:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bc79cad1-a004-4743-888b-0e702b36a275:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 750dddec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 750dddec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 750dddec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3407 (Sat Oct 6 09:30:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3407 (Sat Oct 6 09:30:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "512ff048", "local_conf_timestamp": 3407, "host-ts": 3407}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3404 (Sat Oct 6 09:30:23 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3404 (Sat Oct 6 09:30:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1405530f", "local_conf_timestamp": 3404, "host-ts": 3404}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:63bb20ac-5a36-406e-be57-1075c035119c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:63bb20ac-5a36-406e-be57-1075c035119c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 794fe4f4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 794fe4f4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 794fe4f4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3407 (Sat Oct 6 09:30:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3407 (Sat Oct 6 09:30:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "512ff048", "local_conf_timestamp": 3407, "host-ts": 3407}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3404 (Sat Oct 6 09:30:23 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3404 (Sat Oct 6 09:30:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1405530f", "local_conf_timestamp": 3404, "host-ts": 3404}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:c1557713-869d-4318-a036-fbdccd32c9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1557713-869d-4318-a036-fbdccd32c9bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7c750740 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7c750740 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7c750740 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3417 (Sat Oct 6 09:30:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3417 (Sat Oct 6 09:30:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "107a3a14", "local_conf_timestamp": 3417, "host-ts": 3417}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3414 (Sat Oct 6 09:30:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3414 (Sat Oct 6 09:30:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1043ac0a", "local_conf_timestamp": 3414, "host-ts": 3414}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:d339c7c0-0ef6-47d0-b369-e2bc29b628bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d339c7c0-0ef6-47d0-b369-e2bc29b628bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7f940cdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7f940cdc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7f940cdc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3417 (Sat Oct 6 09:30:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3417 (Sat Oct 6 09:30:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "107a3a14", "local_conf_timestamp": 3417, "host-ts": 3417}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3414 (Sat Oct 6 09:30:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3414 (Sat Oct 6 09:30:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1043ac0a", "local_conf_timestamp": 3414, "host-ts": 3414}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:4a03aa6b-296c-40ba-b112-31133d8b32eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4a03aa6b-296c-40ba-b112-31133d8b32eb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 82ea743e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 82ea743e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 82ea743e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3427 (Sat Oct 6 09:30:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3427 (Sat Oct 6 09:30:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d38464f0", "local_conf_timestamp": 3427, "host-ts": 3427}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3424 (Sat Oct 6 09:30:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3424 (Sat Oct 6 09:30:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c88ad05", "local_conf_timestamp": 3424, "host-ts": 3424}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:4a7239e6-0641-4061-a9e1-705ac71fbdc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4a7239e6-0641-4061-a9e1-705ac71fbdc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8659fdc4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8659fdc4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8659fdc4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3427 (Sat Oct 6 09:30:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3427 (Sat Oct 6 09:30:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d38464f0", "local_conf_timestamp": 3427, "host-ts": 3427}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3424 (Sat Oct 6 09:30:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3424 (Sat Oct 6 09:30:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c88ad05", "local_conf_timestamp": 3424, "host-ts": 3424}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:675b186f-f933-4c72-a848-464ceed95677:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:675b186f-f933-4c72-a848-464ceed95677:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 89779624 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 89779624 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 89779624 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3437 (Sat Oct 6 09:30:58 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3438 (Sat Oct 6 09:30:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "026eb33d", "local_conf_timestamp": 3438, "host-ts": 3437}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3434 (Sat Oct 6 09:30:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3434 (Sat Oct 6 09:30:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18ce5200", "local_conf_timestamp": 3434, "host-ts": 3434}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:500ffb6c-4171-49ac-8f11-636dc32057cd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:500ffb6c-4171-49ac-8f11-636dc32057cd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8cd7ec92 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8cd7ec92 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8cd7ec92 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3437 (Sat Oct 6 09:30:58 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3438 (Sat Oct 6 09:30:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "026eb33d", "local_conf_timestamp": 3438, "host-ts": 3437}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3434 (Sat Oct 6 09:30:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3434 (Sat Oct 6 09:30:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18ce5200", "local_conf_timestamp": 3434, "host-ts": 3434}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:7aeb35e4-0e21-425b-9e0b-db3eb92cfb09:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7aeb35e4-0e21-425b-9e0b-db3eb92cfb09:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 903e89d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 903e89d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 903e89d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3447 (Sat Oct 6 09:31:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3447 (Sat Oct 6 09:31:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8f09df79", "local_conf_timestamp": 3447, "host-ts": 3447}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3444 (Sat Oct 6 09:31:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3445 (Sat Oct 6 09:31:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "72199f8d", "local_conf_timestamp": 3445, "host-ts": 3444}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:0fe988ff-f225-4c70-b20e-d4941d8bda17:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0fe988ff-f225-4c70-b20e-d4941d8bda17:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 945ceb20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 945ceb20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 945ceb20 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3457 (Sat Oct 6 09:31:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3457 (Sat Oct 6 09:31:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ce5c1525", "local_conf_timestamp": 3457, "host-ts": 3457}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3454 (Sat Oct 6 09:31:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3454 (Sat Oct 6 09:31:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0158501e", "local_conf_timestamp": 3454, "host-ts": 3454}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:b5c9a918-654d-4ce7-be1e-286bfc26cf34:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b5c9a918-654d-4ce7-be1e-286bfc26cf34:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 97c6f864 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 97c6f864 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 97c6f864 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3457 (Sat Oct 6 09:31:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3457 (Sat Oct 6 09:31:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ce5c1525", "local_conf_timestamp": 3457, "host-ts": 3457}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3454 (Sat Oct 6 09:31:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3454 (Sat Oct 6 09:31:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0158501e", "local_conf_timestamp": 3454, "host-ts": 3454}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:49e56adb-d202-4589-8416-8164b2821d27:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:49e56adb-d202-4589-8416-8164b2821d27:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9bc30e08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9bc30e08 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9bc30e08 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3467 (Sat Oct 6 09:31:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3467 (Sat Oct 6 09:31:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1a228395", "local_conf_timestamp": 3467, "host-ts": 3467}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3464 (Sat Oct 6 09:31:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3464 (Sat Oct 6 09:31:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d935111", "local_conf_timestamp": 3464, "host-ts": 3464}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:df31cbc7-427e-4dee-a82a-688d53a69dc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:df31cbc7-427e-4dee-a82a-688d53a69dc2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a04dac08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a04dac08 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a04dac08 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3477 (Sat Oct 6 09:31:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3478 (Sat Oct 6 09:31:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cbc85458", "local_conf_timestamp": 3478, "host-ts": 3477}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3474 (Sat Oct 6 09:31:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3474 (Sat Oct 6 09:31:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "09d5ae14", "local_conf_timestamp": 3474, "host-ts": 3474}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:3153a30a-4831-4644-9f65-e62022c548dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3153a30a-4831-4644-9f65-e62022c548dd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a4da2986 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a4da2986 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a4da2986 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3487 (Sat Oct 6 09:31:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3487 (Sat Oct 6 09:31:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2192603f", "local_conf_timestamp": 3487, "host-ts": 3487}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3484 (Sat Oct 6 09:31:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3484 (Sat Oct 6 09:31:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3632ab27", "local_conf_timestamp": 3484, "host-ts": 3484}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:170e0571-d024-4b7f-b125-7679ee0ba1a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:170e0571-d024-4b7f-b125-7679ee0ba1a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a9d7f242 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a9d7f242 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a9d7f242 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3497 (Sat Oct 6 09:31:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3497 (Sat Oct 6 09:31:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "60c7aa63", "local_conf_timestamp": 3497, "host-ts": 3497}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3494 (Sat Oct 6 09:31:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3494 (Sat Oct 6 09:31:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "32745422", "local_conf_timestamp": 3494, "host-ts": 3494}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:2ef1ee03-ac81-4271-869a-1cd1a44c9d7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2ef1ee03-ac81-4271-869a-1cd1a44c9d7a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b008e694 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b008e694 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b008e694 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "646be17f", "local_conf_timestamp": 3504, "host-ts": 3504}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:19008521-0503-40ee-8f84-3f1e3c125d34:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:19008521-0503-40ee-8f84-3f1e3c125d34:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b5616ada on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b5616ada on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b5616ada on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-st\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f368539e410>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda>\n for k, v in _get_he_status(host).items()\n File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 128, in _get_he_status\n raise RuntimeError(\'could not parse JSON: %s\' % ret.out)\nRuntimeError: could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3507 (Sat Oct 6 09:32:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3508 (Sat Oct 6 09:32:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b48c6d98", "local_conf_timestamp": 3508, "host-ts": 3507}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3504 (Sat Oct 6 09:32:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3504 (Sat Oct 6 09:32:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-st\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/377/ Build Number: 377 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/378/ Build Number: 378 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:db705a6c-748a-447b-9bf3-e6df46301cd7:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:db705a6c-748a-447b-9bf3-e6df46301cd7:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running c0e39c02 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c0e39c02 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command c0e39c02 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Sun Oct 7 01:38:45 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4233 (Sun Oct 7 01:38:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "88f742c4", "local_conf_timestamp": 4233, "host-ts": 4233}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Sun Oct 7 01:38:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4234 (Sun Oct 7 01:38:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "88e2a18a", "local_conf_timestamp": 4234, "host-ts": 4234}, "global_maintenance": false} lago.ssh: DEBUG: start task:2b85eb2e-24c2-464c-be4e-3c93db1d2e6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2b85eb2e-24c2-464c-be4e-3c93db1d2e6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c23b3ac4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command c23b3ac4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:a807d3e3-e442-42cf-be9c-cdbf35470925:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a807d3e3-e442-42cf-be9c-cdbf35470925:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c34c9a02 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c34c9a02 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c34c9a02 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Sun Oct 7 01:38:56 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4243 (Sun Oct 7 01:38:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f05088ba", "local_conf_timestamp": 4243, "host-ts": 4243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Sun Oct 7 01:38:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4245 (Sun Oct 7 01:38:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e2356c07", "local_conf_timestamp": 4245, "host-ts": 4244}, "global_maintenance": false} lago.ssh: DEBUG: start task:e9533017-248a-409e-b79c-2c1578806fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e9533017-248a-409e-b79c-2c1578806fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cabe8a02 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cabe8a02 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cabe8a02 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4254 (Sun Oct 7 01:39:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4254 (Sun Oct 7 01:39:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "af5b43e1", "local_conf_timestamp": 4254, "host-ts": 4254}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4254 (Sun Oct 7 01:39:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4254 (Sun Oct 7 01:39:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9174a394", "local_conf_timestamp": 4254, "host-ts": 4254}, "global_maintenance": false} lago.ssh: DEBUG: start task:58ba84d1-15a1-4799-86f2-edbdc68d77c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:58ba84d1-15a1-4799-86f2-edbdc68d77c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cef6155e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cef6155e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cef6155e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8d976629", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9dbfa29b", "local_conf_timestamp": 4264, "host-ts": 4264}, "global_maintenance": false} lago.ssh: DEBUG: start task:7957d4cd-7b19-4356-a8e7-e0e4f18c46ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7957d4cd-7b19-4356-a8e7-e0e4f18c46ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d2471654 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d2471654 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d2471654 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8d976629", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9dbfa29b", "local_conf_timestamp": 4264, "host-ts": 4264}, "global_maintenance": false} lago.ssh: DEBUG: start task:89d99b91-3e05-4054-bead-d8f0eb44fd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:89d99b91-3e05-4054-bead-d8f0eb44fd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d55ff13a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d55ff13a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d55ff13a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4274 (Sun Oct 7 01:39:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4275 (Sun Oct 7 01:39:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54a3687d", "local_conf_timestamp": 4275, "host-ts": 4274}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false} lago.ssh: DEBUG: start task:03e952e2-552c-4106-ba65-28019da139b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:03e952e2-552c-4106-ba65-28019da139b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d83bf174 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d83bf174 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d83bf174 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4274 (Sun Oct 7 01:39:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4275 (Sun Oct 7 01:39:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54a3687d", "local_conf_timestamp": 4275, "host-ts": 4274}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false} lago.ssh: DEBUG: start task:16cde1c5-aa72-4695-aec2-fdb110660f19:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:16cde1c5-aa72-4695-aec2-fdb110660f19:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running db1da734 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command db1da734 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command db1da734 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4284 (Sun Oct 7 01:39:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4284 (Sun Oct 7 01:39:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ae1109f0", "local_conf_timestamp": 4284, "host-ts": 4284}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false} lago.ssh: DEBUG: start task:5bfb5151-f028-4489-8677-d920820cb50c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5bfb5151-f028-4489-8677-d920820cb50c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ddfea4a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ddfea4a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ddfea4a8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4284 (Sun Oct 7 01:39:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4284 (Sun Oct 7 01:39:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ae1109f0", "local_conf_timestamp": 4284, "host-ts": 4284}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false} lago.ssh: DEBUG: start task:6c64d6ae-f7a4-4d85-9b54-8caca9b0b779:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6c64d6ae-f7a4-4d85-9b54-8caca9b0b779:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e0ef624c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e0ef624c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e0ef624c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "00223732", "local_conf_timestamp": 4294, "host-ts": 4294}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "66414abe", "local_conf_timestamp": 4294, "host-ts": 4294}, "global_maintenance": false} lago.ssh: DEBUG: start task:91be1cf1-f052-413e-bf2e-dfc80f2ce1fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:91be1cf1-f052-413e-bf2e-dfc80f2ce1fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e3d20c62 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e3d20c62 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e3d20c62 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "00223732", "local_conf_timestamp": 4294, "host-ts": 4294}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "66414abe", "local_conf_timestamp": 4294, "host-ts": 4294}, "global_maintenance": false} lago.ssh: DEBUG: start task:8da7a617-74ee-4207-a0a4-fe990310dca2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8da7a617-74ee-4207-a0a4-fe990310dca2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e6d722e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e6d722e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e6d722e4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4304 (Sun Oct 7 01:39:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4305 (Sun Oct 7 01:39:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dd1c4c13", "local_conf_timestamp": 4305, "host-ts": 4304}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4304 (Sun Oct 7 01:39:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4304 (Sun Oct 7 01:39:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "305effe3", "local_conf_timestamp": 4304, "host-ts": 4304}, "global_maintenance": false} lago.ssh: DEBUG: start task:b91afc61-0b2c-4eac-9b11-722f765e3ae2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b91afc61-0b2c-4eac-9b11-722f765e3ae2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ea23c182 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ea23c182 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ea23c182 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4304 (Sun Oct 7 01:39:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4305 (Sun Oct 7 01:39:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dd1c4c13", "local_conf_timestamp": 4305, "host-ts": 4304}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4304 (Sun Oct 7 01:39:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4304 (Sun Oct 7 01:39:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "305effe3", "local_conf_timestamp": 4304, "host-ts": 4304}, "global_maintenance": false} lago.ssh: DEBUG: start task:6fd22ca3-ff2e-4b0c-a7db-16515e9fd894:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6fd22ca3-ff2e-4b0c-a7db-16515e9fd894:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ed33fcf2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ed33fcf2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ed33fcf2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2cf1fcda", "local_conf_timestamp": 4314, "host-ts": 4314}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "341800e6", "local_conf_timestamp": 4314, "host-ts": 4314}, "global_maintenance": false} lago.ssh: DEBUG: start task:c959b4a8-c6f0-406b-8597-07f8364940b6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c959b4a8-c6f0-406b-8597-07f8364940b6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee40613a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ee40613a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ee40613a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2cf1fcda", "local_conf_timestamp": 4314, "host-ts": 4314}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "341800e6", "local_conf_timestamp": 4314, "host-ts": 4314}, "global_maintenance": false} lago.ssh: DEBUG: start task:53853219-0b30-456c-bcd4-ebbfbd2fc3f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:53853219-0b30-456c-bcd4-ebbfbd2fc3f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f54ac0ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f54ac0ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f54ac0ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4324 (Sun Oct 7 01:40:17 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4325 (Sun Oct 7 01:40:17 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "2f0dbd6e", "local_conf_timestamp": 4325, "host-ts": 4324}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4325 (Sun Oct 7 01:40:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4325 (Sun Oct 7 01:40:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd62238f", "local_conf_timestamp": 4325, "host-ts": 4325}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:287a25d7-2123-4ad8-a26b-32c2c460170d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:287a25d7-2123-4ad8-a26b-32c2c460170d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a929c50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none lago.ssh: DEBUG: Command a929c50e on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for Engine Maintenance to reset... lago.ssh: DEBUG: start task:a4d502a4-4af2-4295-973b-b4796a6266d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a4d502a4-4af2-4295-973b-b4796a6266d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ab734e20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ab734e20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ab734e20 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4634 (Sun Oct 7 01:45:27 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "7d32e4c7", "local_conf_timestamp": 4635, "host-ts": 4634}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4635 (Sun Oct 7 01:45:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "20c50afa", "local_conf_timestamp": 4635, "host-ts": 4635}, "global_maintenance": false} lago.ssh: DEBUG: start task:4bf4be88-50f2-41e1-b6fa-f2f8b3915d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4bf4be88-50f2-41e1-b6fa-f2f8b3915d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ac83f06c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ac83f06c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ac83f06c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4634 (Sun Oct 7 01:45:27 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "7d32e4c7", "local_conf_timestamp": 4635, "host-ts": 4634}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4635 (Sun Oct 7 01:45:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "20c50afa", "local_conf_timestamp": 4635, "host-ts": 4635}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 130, in local_maintenance _wait_for_engine_maintenance(host, he_index, False) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 73, in _wait_for_engine_maintenance lambda: _is_state_maintenance(host, "LocalMaintenance") is False File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:db705a6c-748a-447b-9bf3-e6df46301cd7:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:db705a6c-748a-447b-9bf3-e6df46301cd7:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running c0e39c02 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c0e39c02 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command c0e39c02 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Sun Oct 7 01:38:45 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4233 (Sun Oct 7 01:38:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "88f742c4", "local_conf_timestamp": 4233, "host-ts": 4233}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Sun Oct 7 01:38:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4234 (Sun Oct 7 01:38:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "88e2a18a", "local_conf_timestamp": 4234, "host-ts": 4234}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2b85eb2e-24c2-464c-be4e-3c93db1d2e6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2b85eb2e-24c2-464c-be4e-3c93db1d2e6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c23b3ac4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command c23b3ac4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:a807d3e3-e442-42cf-be9c-cdbf35470925:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a807d3e3-e442-42cf-be9c-cdbf35470925:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c34c9a02 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c34c9a02 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c34c9a02 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Sun Oct 7 01:38:56 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4243 (Sun Oct 7 01:38:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f05088ba", "local_conf_timestamp": 4243, "host-ts": 4243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Sun Oct 7 01:38:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4245 (Sun Oct 7 01:38:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e2356c07", "local_conf_timestamp": 4245, "host-ts": 4244}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e9533017-248a-409e-b79c-2c1578806fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e9533017-248a-409e-b79c-2c1578806fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cabe8a02 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cabe8a02 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cabe8a02 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4254 (Sun Oct 7 01:39:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4254 (Sun Oct 7 01:39:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "af5b43e1", "local_conf_timestamp": 4254, "host-ts": 4254}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4254 (Sun Oct 7 01:39:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4254 (Sun Oct 7 01:39:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9174a394", "local_conf_timestamp": 4254, "host-ts": 4254}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:58ba84d1-15a1-4799-86f2-edbdc68d77c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:58ba84d1-15a1-4799-86f2-edbdc68d77c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cef6155e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cef6155e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cef6155e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8d976629", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9dbfa29b", "local_conf_timestamp": 4264, "host-ts": 4264}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7957d4cd-7b19-4356-a8e7-e0e4f18c46ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7957d4cd-7b19-4356-a8e7-e0e4f18c46ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d2471654 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d2471654 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d2471654 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8d976629", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4264 (Sun Oct 7 01:39:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4264 (Sun Oct 7 01:39:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9dbfa29b", "local_conf_timestamp": 4264, "host-ts": 4264}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:89d99b91-3e05-4054-bead-d8f0eb44fd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:89d99b91-3e05-4054-bead-d8f0eb44fd2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d55ff13a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d55ff13a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d55ff13a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4274 (Sun Oct 7 01:39:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4275 (Sun Oct 7 01:39:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54a3687d", "local_conf_timestamp": 4275, "host-ts": 4274}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:03e952e2-552c-4106-ba65-28019da139b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:03e952e2-552c-4106-ba65-28019da139b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d83bf174 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d83bf174 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d83bf174 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4274 (Sun Oct 7 01:39:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4275 (Sun Oct 7 01:39:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54a3687d", "local_conf_timestamp": 4275, "host-ts": 4274}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:16cde1c5-aa72-4695-aec2-fdb110660f19:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:16cde1c5-aa72-4695-aec2-fdb110660f19:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running db1da734 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command db1da734 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command db1da734 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4284 (Sun Oct 7 01:39:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4284 (Sun Oct 7 01:39:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ae1109f0", "local_conf_timestamp": 4284, "host-ts": 4284}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5bfb5151-f028-4489-8677-d920820cb50c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5bfb5151-f028-4489-8677-d920820cb50c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ddfea4a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ddfea4a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ddfea4a8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4284 (Sun Oct 7 01:39:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4284 (Sun Oct 7 01:39:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ae1109f0", "local_conf_timestamp": 4284, "host-ts": 4284}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4274 (Sun Oct 7 01:39:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4274 (Sun Oct 7 01:39:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eec46806", "local_conf_timestamp": 4274, "host-ts": 4274}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6c64d6ae-f7a4-4d85-9b54-8caca9b0b779:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6c64d6ae-f7a4-4d85-9b54-8caca9b0b779:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e0ef624c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e0ef624c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e0ef624c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "00223732", "local_conf_timestamp": 4294, "host-ts": 4294}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "66414abe", "local_conf_timestamp": 4294, "host-ts": 4294}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:91be1cf1-f052-413e-bf2e-dfc80f2ce1fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:91be1cf1-f052-413e-bf2e-dfc80f2ce1fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e3d20c62 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e3d20c62 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e3d20c62 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "00223732", "local_conf_timestamp": 4294, "host-ts": 4294}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4294 (Sun Oct 7 01:39:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4294 (Sun Oct 7 01:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "66414abe", "local_conf_timestamp": 4294, "host-ts": 4294}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8da7a617-74ee-4207-a0a4-fe990310dca2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8da7a617-74ee-4207-a0a4-fe990310dca2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e6d722e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e6d722e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e6d722e4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4304 (Sun Oct 7 01:39:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4305 (Sun Oct 7 01:39:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dd1c4c13", "local_conf_timestamp": 4305, "host-ts": 4304}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4304 (Sun Oct 7 01:39:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4304 (Sun Oct 7 01:39:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "305effe3", "local_conf_timestamp": 4304, "host-ts": 4304}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b91afc61-0b2c-4eac-9b11-722f765e3ae2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b91afc61-0b2c-4eac-9b11-722f765e3ae2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ea23c182 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ea23c182 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ea23c182 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4304 (Sun Oct 7 01:39:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4305 (Sun Oct 7 01:39:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dd1c4c13", "local_conf_timestamp": 4305, "host-ts": 4304}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4304 (Sun Oct 7 01:39:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4304 (Sun Oct 7 01:39:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "305effe3", "local_conf_timestamp": 4304, "host-ts": 4304}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6fd22ca3-ff2e-4b0c-a7db-16515e9fd894:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6fd22ca3-ff2e-4b0c-a7db-16515e9fd894:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ed33fcf2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ed33fcf2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ed33fcf2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2cf1fcda", "local_conf_timestamp": 4314, "host-ts": 4314}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "341800e6", "local_conf_timestamp": 4314, "host-ts": 4314}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c959b4a8-c6f0-406b-8597-07f8364940b6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c959b4a8-c6f0-406b-8597-07f8364940b6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee40613a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ee40613a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ee40613a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2cf1fcda", "local_conf_timestamp": 4314, "host-ts": 4314}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4314 (Sun Oct 7 01:40:06 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4314 (Sun Oct 7 01:40:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "341800e6", "local_conf_timestamp": 4314, "host-ts": 4314}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:53853219-0b30-456c-bcd4-ebbfbd2fc3f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:53853219-0b30-456c-bcd4-ebbfbd2fc3f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f54ac0ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f54ac0ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f54ac0ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4324 (Sun Oct 7 01:40:17 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4325 (Sun Oct 7 01:40:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "2f0dbd6e", "local_conf_timestamp": 4325, "host-ts": 4324}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4325 (Sun Oct 7 01:40:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4325 (Sun Oct 7 01:40:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd62238f", "local_conf_timestamp": 4325, "host-ts": 4325}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:287a25d7-2123-4ad8-a26b-32c2c460170d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:287a25d7-2123-4ad8-a26b-32c2c460170d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a929c50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none\nlago.ssh: DEBUG: Command a929c50e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for Engine Maintenance to reset...\nlago.ssh: DEBUG: start task:a4d502a4-4af2-4295-973b-b4796a6266d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a4d502a4-4af2-4295-973b-b4796a6266d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ab734e20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ab734e20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ab734e20 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4634 (Sun Oct 7 01:45:27 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "7d32e4c7", "local_conf_timestamp": 4635, "host-ts": 4634}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4635 (Sun Oct 7 01:45:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "20c50afa", "local_conf_timestamp": 4635, "host-ts": 4635}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4bf4be88-50f2-41e1-b6fa-f2f8b3915d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4bf4be88-50f2-41e1-b6fa-f2f8b3915d32:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ac83f06c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ac83f06c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ac83f06c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4634 (Sun Oct 7 01:45:27 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "7d32e4c7", "local_conf_timestamp": 4635, "host-ts": 4634}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4635 (Sun Oct 7 01:45:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4635 (Sun Oct 7 01:45:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "20c50afa", "local_conf_timestamp": 4635, "host-ts": 4635}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/379/ Build Number: 379 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:9d557828-8c43-4bee-9608-4fedfbda0a42:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:9d557828-8c43-4bee-9608-4fedfbda0a42:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running ac9509f4 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ac9509f4 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command ac9509f4 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sun Oct 7 09:44:57 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4175 (Sun Oct 7 09:44:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e79ba1a", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Oct 7 09:45:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Sun Oct 7 09:45:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ab749ebb", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:5a2143bc-961e-451b-a4d8-adab3fe4cc29:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5a2143bc-961e-451b-a4d8-adab3fe4cc29:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ada5d3be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command ada5d3be on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:230db887-044d-40bd-a5e1-63376e562d16:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:230db887-044d-40bd-a5e1-63376e562d16:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aec884ee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aec884ee on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command aec884ee on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Oct 7 09:45:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4185 (Sun Oct 7 09:45:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73516ed9", "local_conf_timestamp": 4185, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Oct 7 09:45:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Sun Oct 7 09:45:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ab749ebb", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:afea5971-001f-4f32-a531-d8ff4cfffcf8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:afea5971-001f-4f32-a531-d8ff4cfffcf8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b6485884 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b6485884 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b6485884 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Oct 7 09:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4195 (Sun Oct 7 09:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c5979ca", "local_conf_timestamp": 4195, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Oct 7 09:45:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4195 (Sun Oct 7 09:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af3261be", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:c7cc6be6-09a7-4c90-90cb-41994b35f692:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c7cc6be6-09a7-4c90-90cb-41994b35f692:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ba2f43ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ba2f43ae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ba2f43ae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Oct 7 09:45:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4206 (Sun Oct 7 09:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4823575", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Oct 7 09:45:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Sun Oct 7 09:45:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:cb241c64-ea29-466b-9e83-b3710e2b3ba2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cb241c64-ea29-466b-9e83-b3710e2b3ba2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bd489fae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bd489fae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bd489fae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Oct 7 09:45:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4206 (Sun Oct 7 09:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4823575", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Oct 7 09:45:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Sun Oct 7 09:45:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in _wait_for_engine_migration testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 246, in assert_equals_within time.sleep(3) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:9d557828-8c43-4bee-9608-4fedfbda0a42:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:9d557828-8c43-4bee-9608-4fedfbda0a42:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running ac9509f4 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ac9509f4 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command ac9509f4 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sun Oct 7 09:44:57 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4175 (Sun Oct 7 09:44:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5e79ba1a", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Oct 7 09:45:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Sun Oct 7 09:45:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ab749ebb", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5a2143bc-961e-451b-a4d8-adab3fe4cc29:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5a2143bc-961e-451b-a4d8-adab3fe4cc29:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ada5d3be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command ada5d3be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:230db887-044d-40bd-a5e1-63376e562d16:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:230db887-044d-40bd-a5e1-63376e562d16:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aec884ee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aec884ee on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command aec884ee on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Oct 7 09:45:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4185 (Sun Oct 7 09:45:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73516ed9", "local_conf_timestamp": 4185, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Oct 7 09:45:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Sun Oct 7 09:45:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ab749ebb", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:afea5971-001f-4f32-a531-d8ff4cfffcf8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:afea5971-001f-4f32-a531-d8ff4cfffcf8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b6485884 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b6485884 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b6485884 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Oct 7 09:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4195 (Sun Oct 7 09:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c5979ca", "local_conf_timestamp": 4195, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Oct 7 09:45:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4195 (Sun Oct 7 09:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af3261be", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c7cc6be6-09a7-4c90-90cb-41994b35f692:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c7cc6be6-09a7-4c90-90cb-41994b35f692:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ba2f43ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ba2f43ae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ba2f43ae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Oct 7 09:45:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4206 (Sun Oct 7 09:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4823575", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Oct 7 09:45:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Sun Oct 7 09:45:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cb241c64-ea29-466b-9e83-b3710e2b3ba2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cb241c64-ea29-466b-9e83-b3710e2b3ba2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bd489fae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bd489fae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bd489fae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Oct 7 09:45:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4206 (Sun Oct 7 09:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4823575", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Oct 7 09:45:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Sun Oct 7 09:45:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/380/ Build Number: 380 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:b8cfb2bb-fa79-4545-8e42-d5bce1f3a0df:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:b8cfb2bb-fa79-4545-8e42-d5bce1f3a0df:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 5400bf52 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5400bf52 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 5400bf52 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Sun Oct 7 17:42:00 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4226 (Sun Oct 7 17:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4d7938ae", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Sun Oct 7 17:42:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4233 (Sun Oct 7 17:42:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef0fde9a", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:c63fdbdd-51e2-4caa-8f73-b7b4284de68e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c63fdbdd-51e2-4caa-8f73-b7b4284de68e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 550cad98 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 550cad98 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:a69f0386-565a-4f6f-95e8-7342a00cb473:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a69f0386-565a-4f6f-95e8-7342a00cb473:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 561d1a56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 561d1a56 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 561d1a56 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Sun Oct 7 17:42:11 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4236 (Sun Oct 7 17:42:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "911e7891", "local_conf_timestamp": 4236, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Sun Oct 7 17:42:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4243 (Sun Oct 7 17:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f2df2381", "local_conf_timestamp": 4243, "host-ts": 4243}, "global_maintenance": false} lago.ssh: DEBUG: start task:90260600-c634-42d7-abe5-5938a7bbac41:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:90260600-c634-42d7-abe5-5938a7bbac41:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5d285996 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5d285996 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5d285996 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Sun Oct 7 17:42:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Sun Oct 7 17:42:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7b9b15ed", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Sun Oct 7 17:42:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4243 (Sun Oct 7 17:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f2df2381", "local_conf_timestamp": 4243, "host-ts": 4243}, "global_maintenance": false} lago.ssh: DEBUG: start task:06f8935f-9554-4fb9-afb8-4de9a25ee2ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:06f8935f-9554-4fb9-afb8-4de9a25ee2ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 605cbe04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 605cbe04 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 605cbe04 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Sun Oct 7 17:42:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Sun Oct 7 17:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Sun Oct 7 17:42:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4253 (Sun Oct 7 17:42:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f699dc84", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false} lago.ssh: DEBUG: start task:d8e75817-7732-4eae-909f-8d7050072bfb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d8e75817-7732-4eae-909f-8d7050072bfb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6339f1be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6339f1be on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6339f1be on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Sun Oct 7 17:42:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Sun Oct 7 17:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Sun Oct 7 17:42:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4253 (Sun Oct 7 17:42:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f699dc84", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false} lago.ssh: DEBUG: start task:f497725b-2fcc-4538-8498-4e5e7527e105:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f497725b-2fcc-4538-8498-4e5e7527e105:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 66500d16 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 66500d16 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 66500d16 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Sun Oct 7 17:42:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Sun Oct 7 17:42:42 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Sun Oct 7 17:42:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4263 (Sun Oct 7 17:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d6fe813", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false} lago.ssh: DEBUG: start task:f4d23cf8-7f34-4572-90b8-feefe04e2863:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f4d23cf8-7f34-4572-90b8-feefe04e2863:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 698622ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 698622ea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 698622ea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Sun Oct 7 17:42:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4278 (Sun Oct 7 17:42:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Sun Oct 7 17:42:47 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4273 (Sun Oct 7 17:42:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89291716", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false} lago.ssh: DEBUG: start task:4f4b9a03-7b53-421f-ab65-11678404fb8f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4f4b9a03-7b53-421f-ab65-11678404fb8f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6d4a8b1e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6d4a8b1e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6d4a8b1e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Sun Oct 7 17:42:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4278 (Sun Oct 7 17:42:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Sun Oct 7 17:42:47 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4273 (Sun Oct 7 17:42:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89291716", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false} lago.ssh: DEBUG: start task:1ebd61b0-571a-4a97-9983-d2fa0351e852:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1ebd61b0-571a-4a97-9983-d2fa0351e852:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71357e32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71357e32 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71357e32 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4288 (Sun Oct 7 17:43:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4289 (Sun Oct 7 17:43:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "99c64800", "local_conf_timestamp": 4289, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4283 (Sun Oct 7 17:42:58 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4284 (Sun Oct 7 17:42:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false} lago.ssh: DEBUG: start task:0a0c89ad-97b1-4a0f-9b6c-1462696ef291:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0a0c89ad-97b1-4a0f-9b6c-1462696ef291:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 74223c98 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 74223c98 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 74223c98 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4288 (Sun Oct 7 17:43:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4289 (Sun Oct 7 17:43:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "99c64800", "local_conf_timestamp": 4289, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4283 (Sun Oct 7 17:42:58 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4284 (Sun Oct 7 17:42:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false} lago.ssh: DEBUG: start task:36091533-ce94-4ebe-9623-fc1e514c4695:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:36091533-ce94-4ebe-9623-fc1e514c4695:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7717718e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7717718e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7717718e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4298 (Sun Oct 7 17:43:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4298 (Sun Oct 7 17:43:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40f24654", "local_conf_timestamp": 4298, "host-ts": 4298}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4293 (Sun Oct 7 17:43:07 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4293 (Sun Oct 7 17:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "01ac35ae", "local_conf_timestamp": 4293, "host-ts": 4293}, "global_maintenance": false} lago.ssh: DEBUG: start task:314da6f5-63da-45ae-b4df-1b4442673890:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:314da6f5-63da-45ae-b4df-1b4442673890:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7a1d7b1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7a1d7b1c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7a1d7b1c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4298 (Sun Oct 7 17:43:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4298 (Sun Oct 7 17:43:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40f24654", "local_conf_timestamp": 4298, "host-ts": 4298}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4293 (Sun Oct 7 17:43:07 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4293 (Sun Oct 7 17:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "01ac35ae", "local_conf_timestamp": 4293, "host-ts": 4293}, "global_maintenance": false} lago.ssh: DEBUG: start task:c511e69f-ffbf-4d84-bfac-bb2a5e7a202c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c511e69f-ffbf-4d84-bfac-bb2a5e7a202c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7d4122da on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7d4122da on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7d4122da on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4308 (Sun Oct 7 17:43:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4309 (Sun Oct 7 17:43:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d7207b06", "local_conf_timestamp": 4309, "host-ts": 4308}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4303 (Sun Oct 7 17:43:17 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4303 (Sun Oct 7 17:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "57b380f3", "local_conf_timestamp": 4303, "host-ts": 4303}, "global_maintenance": false} lago.ssh: DEBUG: start task:cb33373b-3c4e-4728-9cdc-9001ea90daa8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cb33373b-3c4e-4728-9cdc-9001ea90daa8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7e52401e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7e52401e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7e52401e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4308 (Sun Oct 7 17:43:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4309 (Sun Oct 7 17:43:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d7207b06", "local_conf_timestamp": 4309, "host-ts": 4308}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4303 (Sun Oct 7 17:43:17 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4303 (Sun Oct 7 17:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "57b380f3", "local_conf_timestamp": 4303, "host-ts": 4303}, "global_maintenance": false} lago.ssh: DEBUG: start task:6174dac1-840f-4a75-bd89-bd2042963be5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6174dac1-840f-4a75-bd89-bd2042963be5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 855f408c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 855f408c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 855f408c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4318 (Sun Oct 7 17:43:32 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4318 (Sun Oct 7 17:43:32 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "13e39e09", "local_conf_timestamp": 4318, "host-ts": 4318}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4313 (Sun Oct 7 17:43:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4313 (Sun Oct 7 17:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "53f4c9e9", "local_conf_timestamp": 4313, "host-ts": 4313}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:b8cfb2bb-fa79-4545-8e42-d5bce1f3a0df:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:b8cfb2bb-fa79-4545-8e42-d5bce1f3a0df:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 5400bf52 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5400bf52 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 5400bf52 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Sun Oct 7 17:42:00 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4226 (Sun Oct 7 17:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4d7938ae", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Sun Oct 7 17:42:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4233 (Sun Oct 7 17:42:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef0fde9a", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c63fdbdd-51e2-4caa-8f73-b7b4284de68e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c63fdbdd-51e2-4caa-8f73-b7b4284de68e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 550cad98 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 550cad98 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:a69f0386-565a-4f6f-95e8-7342a00cb473:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a69f0386-565a-4f6f-95e8-7342a00cb473:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 561d1a56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 561d1a56 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 561d1a56 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Sun Oct 7 17:42:11 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4236 (Sun Oct 7 17:42:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "911e7891", "local_conf_timestamp": 4236, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Sun Oct 7 17:42:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4243 (Sun Oct 7 17:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f2df2381", "local_conf_timestamp": 4243, "host-ts": 4243}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:90260600-c634-42d7-abe5-5938a7bbac41:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:90260600-c634-42d7-abe5-5938a7bbac41:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5d285996 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5d285996 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5d285996 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Sun Oct 7 17:42:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Sun Oct 7 17:42:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7b9b15ed", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Sun Oct 7 17:42:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4243 (Sun Oct 7 17:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f2df2381", "local_conf_timestamp": 4243, "host-ts": 4243}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:06f8935f-9554-4fb9-afb8-4de9a25ee2ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:06f8935f-9554-4fb9-afb8-4de9a25ee2ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 605cbe04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 605cbe04 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 605cbe04 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Sun Oct 7 17:42:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Sun Oct 7 17:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Sun Oct 7 17:42:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4253 (Sun Oct 7 17:42:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f699dc84", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d8e75817-7732-4eae-909f-8d7050072bfb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d8e75817-7732-4eae-909f-8d7050072bfb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6339f1be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6339f1be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6339f1be on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Sun Oct 7 17:42:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Sun Oct 7 17:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Sun Oct 7 17:42:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4253 (Sun Oct 7 17:42:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f699dc84", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f497725b-2fcc-4538-8498-4e5e7527e105:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f497725b-2fcc-4538-8498-4e5e7527e105:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 66500d16 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 66500d16 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 66500d16 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Sun Oct 7 17:42:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Sun Oct 7 17:42:42 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Sun Oct 7 17:42:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4263 (Sun Oct 7 17:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d6fe813", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f4d23cf8-7f34-4572-90b8-feefe04e2863:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f4d23cf8-7f34-4572-90b8-feefe04e2863:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 698622ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 698622ea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 698622ea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Sun Oct 7 17:42:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4278 (Sun Oct 7 17:42:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Sun Oct 7 17:42:47 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4273 (Sun Oct 7 17:42:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89291716", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4f4b9a03-7b53-421f-ab65-11678404fb8f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4f4b9a03-7b53-421f-ab65-11678404fb8f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6d4a8b1e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6d4a8b1e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6d4a8b1e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Sun Oct 7 17:42:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4278 (Sun Oct 7 17:42:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Sun Oct 7 17:42:47 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4273 (Sun Oct 7 17:42:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89291716", "local_conf_timestamp": 4273, "host-ts": 4273}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1ebd61b0-571a-4a97-9983-d2fa0351e852:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1ebd61b0-571a-4a97-9983-d2fa0351e852:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71357e32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71357e32 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71357e32 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4288 (Sun Oct 7 17:43:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4289 (Sun Oct 7 17:43:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "99c64800", "local_conf_timestamp": 4289, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4283 (Sun Oct 7 17:42:58 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4284 (Sun Oct 7 17:42:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0a0c89ad-97b1-4a0f-9b6c-1462696ef291:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0a0c89ad-97b1-4a0f-9b6c-1462696ef291:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 74223c98 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 74223c98 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 74223c98 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4288 (Sun Oct 7 17:43:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4289 (Sun Oct 7 17:43:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "99c64800", "local_conf_timestamp": 4289, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4283 (Sun Oct 7 17:42:58 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4284 (Sun Oct 7 17:42:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9b8e5f08", "local_conf_timestamp": 4284, "host-ts": 4283}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:36091533-ce94-4ebe-9623-fc1e514c4695:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:36091533-ce94-4ebe-9623-fc1e514c4695:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7717718e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7717718e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7717718e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4298 (Sun Oct 7 17:43:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4298 (Sun Oct 7 17:43:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40f24654", "local_conf_timestamp": 4298, "host-ts": 4298}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4293 (Sun Oct 7 17:43:07 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4293 (Sun Oct 7 17:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "01ac35ae", "local_conf_timestamp": 4293, "host-ts": 4293}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:314da6f5-63da-45ae-b4df-1b4442673890:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:314da6f5-63da-45ae-b4df-1b4442673890:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7a1d7b1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7a1d7b1c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7a1d7b1c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4298 (Sun Oct 7 17:43:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4298 (Sun Oct 7 17:43:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40f24654", "local_conf_timestamp": 4298, "host-ts": 4298}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4293 (Sun Oct 7 17:43:07 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4293 (Sun Oct 7 17:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "01ac35ae", "local_conf_timestamp": 4293, "host-ts": 4293}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c511e69f-ffbf-4d84-bfac-bb2a5e7a202c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c511e69f-ffbf-4d84-bfac-bb2a5e7a202c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7d4122da on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7d4122da on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7d4122da on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4308 (Sun Oct 7 17:43:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4309 (Sun Oct 7 17:43:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d7207b06", "local_conf_timestamp": 4309, "host-ts": 4308}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4303 (Sun Oct 7 17:43:17 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4303 (Sun Oct 7 17:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "57b380f3", "local_conf_timestamp": 4303, "host-ts": 4303}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cb33373b-3c4e-4728-9cdc-9001ea90daa8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cb33373b-3c4e-4728-9cdc-9001ea90daa8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7e52401e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7e52401e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7e52401e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4308 (Sun Oct 7 17:43:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4309 (Sun Oct 7 17:43:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d7207b06", "local_conf_timestamp": 4309, "host-ts": 4308}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4303 (Sun Oct 7 17:43:17 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4303 (Sun Oct 7 17:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "57b380f3", "local_conf_timestamp": 4303, "host-ts": 4303}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6174dac1-840f-4a75-bd89-bd2042963be5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6174dac1-840f-4a75-bd89-bd2042963be5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 855f408c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 855f408c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 855f408c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4318 (Sun Oct 7 17:43:32 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4318 (Sun Oct 7 17:43:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "13e39e09", "local_conf_timestamp": 4318, "host-ts": 4318}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4313 (Sun Oct 7 17:43:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4313 (Sun Oct 7 17:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "53f4c9e9", "local_conf_timestamp": 4313, "host-ts": 4313}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/381/ Build Number: 381 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running a1d1682e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Mon Oct 8 01:43:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4198 (Mon Oct 8 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3ce78604", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Mon Oct 8 01:43:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4206 (Mon Oct 8 01:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a30622a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command a30622a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a44300ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Mon Oct 8 01:43:56 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4208 (Mon Oct 8 01:43:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5b00040", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Mon Oct 8 01:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4216 (Mon Oct 8 01:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "60ac78cd", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ab57ee0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ad2bd33a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b483bd00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4239 (Mon Oct 8 01:44:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4240 (Mon Oct 8 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "62e27ecd", "local_conf_timestamp": 4240, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Mon Oct 8 01:44:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4237 (Mon Oct 8 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "681b83c9", "local_conf_timestamp": 4237, "host-ts": 4236}, "global_maintenance": false} lago.ssh: DEBUG: start task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b7b73a60 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false} lago.ssh: DEBUG: start task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running baa81564 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false} lago.ssh: DEBUG: start task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running be1b6e9e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false} lago.ssh: DEBUG: start task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c0fcb4ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false} lago.ssh: DEBUG: start task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c4258ed2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false} lago.ssh: DEBUG: start task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c823eace on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false} lago.ssh: DEBUG: start task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb0bd06c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false} lago.ssh: DEBUG: start task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cde223cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false} lago.ssh: DEBUG: start task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d0baaae2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4290 (Mon Oct 8 01:45:19 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4290 (Mon Oct 8 01:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "62b65a31", "local_conf_timestamp": 4290, "host-ts": 4290}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4286 (Mon Oct 8 01:45:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4286 (Mon Oct 8 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running a1d1682e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Mon Oct 8 01:43:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4198 (Mon Oct 8 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3ce78604", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Mon Oct 8 01:43:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4206 (Mon Oct 8 01:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a30622a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command a30622a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a44300ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Mon Oct 8 01:43:56 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4208 (Mon Oct 8 01:43:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5b00040", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Mon Oct 8 01:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4216 (Mon Oct 8 01:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "60ac78cd", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ab57ee0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ad2bd33a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b483bd00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4239 (Mon Oct 8 01:44:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4240 (Mon Oct 8 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "62e27ecd", "local_conf_timestamp": 4240, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Mon Oct 8 01:44:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4237 (Mon Oct 8 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "681b83c9", "local_conf_timestamp": 4237, "host-ts": 4236}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b7b73a60 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running baa81564 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running be1b6e9e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c0fcb4ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c4258ed2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c823eace on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb0bd06c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cde223cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d0baaae2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4290 (Mon Oct 8 01:45:19 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4290 (Mon Oct 8 01:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "62b65a31", "local_conf_timestamp": 4290, "host-ts": 4290}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4286 (Mon Oct 8 01:45:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4286 (Mon Oct 8 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/382/ Build Number: 382 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/383/ Build Number: 383 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/384/ Build Number: 384 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:3c73e9c4-b309-40fb-b1fa-ed36da927d07:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:3c73e9c4-b309-40fb-b1fa-ed36da927d07:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 24a15502 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 24a15502 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 24a15502 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Tue Oct 9 01:39:06 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4217 (Tue Oct 9 01:39:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "43ec0e80", "local_conf_timestamp": 4217, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Tue Oct 9 01:39:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4225 (Tue Oct 9 01:39:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:1fcdbfe0-01de-40ae-bf85-7a17775da330:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1fcdbfe0-01de-40ae-bf85-7a17775da330:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 25aded20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 25aded20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:f3a614d4-1caf-4671-8ebc-2dfde68a8388:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f3a614d4-1caf-4671-8ebc-2dfde68a8388:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 26c47904 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 26c47904 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 26c47904 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Tue Oct 9 01:39:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4227 (Tue Oct 9 01:39:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fc34c880", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Tue Oct 9 01:39:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4225 (Tue Oct 9 01:39:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:9fcf021e-6d26-468f-b607-a3670ee1eaaf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9fcf021e-6d26-468f-b607-a3670ee1eaaf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2e05fc88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2e05fc88 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2e05fc88 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Tue Oct 9 01:39:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4248 (Tue Oct 9 01:39:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67b39f04", "local_conf_timestamp": 4248, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Tue Oct 9 01:39:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4245 (Tue Oct 9 01:39:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7fd6798f", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:f01bca9b-8543-4d58-939b-b3e24fff320b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f01bca9b-8543-4d58-939b-b3e24fff320b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 30f91402 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 30f91402 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 30f91402 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Tue Oct 9 01:39:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4248 (Tue Oct 9 01:39:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67b39f04", "local_conf_timestamp": 4248, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Tue Oct 9 01:39:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4245 (Tue Oct 9 01:39:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7fd6798f", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:1cd91097-ce30-425f-856c-73f7581cc7bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1cd91097-ce30-425f-856c-73f7581cc7bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 34323e5a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 34323e5a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 34323e5a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Tue Oct 9 01:39:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Tue Oct 9 01:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Tue Oct 9 01:39:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Tue Oct 9 01:39:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} lago.ssh: DEBUG: start task:3aa0fe5c-5836-4e7b-9c7b-8aa0d676807d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3aa0fe5c-5836-4e7b-9c7b-8aa0d676807d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 371b6650 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 371b6650 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 371b6650 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Tue Oct 9 01:39:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Tue Oct 9 01:39:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Tue Oct 9 01:39:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Tue Oct 9 01:39:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} lago.ssh: DEBUG: start task:cb267a07-39bb-4c78-aca2-d1d0652d6c7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cb267a07-39bb-4c78-aca2-d1d0652d6c7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3a5cb1b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3a5cb1b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3a5cb1b6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Tue Oct 9 01:39:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4267 (Tue Oct 9 01:39:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "701bf950", "local_conf_timestamp": 4267, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4265 (Tue Oct 9 01:39:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4265 (Tue Oct 9 01:39:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775b8785", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false} lago.ssh: DEBUG: start task:1da66f14-fc55-4fdf-a10f-bbd07b7dc01e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1da66f14-fc55-4fdf-a10f-bbd07b7dc01e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3d5da38e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3d5da38e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3d5da38e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Tue Oct 9 01:39:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4267 (Tue Oct 9 01:39:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "701bf950", "local_conf_timestamp": 4267, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4265 (Tue Oct 9 01:39:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4265 (Tue Oct 9 01:39:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775b8785", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false} lago.ssh: DEBUG: start task:489d8ae3-b227-4acc-8a31-9d0574f90918:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:489d8ae3-b227-4acc-8a31-9d0574f90918:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 40834a0a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 40834a0a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 40834a0a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Tue Oct 9 01:40:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4277 (Tue Oct 9 01:40:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4275 (Tue Oct 9 01:40:04 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4275 (Tue Oct 9 01:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false} lago.ssh: DEBUG: start task:55a1f1ed-a80e-4e19-87b5-1e32bc97e4c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:55a1f1ed-a80e-4e19-87b5-1e32bc97e4c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 439d8a48 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 439d8a48 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 439d8a48 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Tue Oct 9 01:40:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4277 (Tue Oct 9 01:40:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4275 (Tue Oct 9 01:40:04 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4275 (Tue Oct 9 01:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false} lago.ssh: DEBUG: start task:7268f134-03c9-456b-b9a5-042f135adc89:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7268f134-03c9-456b-b9a5-042f135adc89:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 46bdf672 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 46bdf672 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 46bdf672 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Tue Oct 9 01:40:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4287 (Tue Oct 9 01:40:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "539d9689", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4285 (Tue Oct 9 01:40:14 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4285 (Tue Oct 9 01:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffdea53d", "local_conf_timestamp": 4285, "host-ts": 4285}, "global_maintenance": false} lago.ssh: DEBUG: start task:5b5ddba0-e48c-4425-b001-ee2a99288230:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5b5ddba0-e48c-4425-b001-ee2a99288230:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4a0f5500 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4a0f5500 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4a0f5500 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Tue Oct 9 01:40:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4287 (Tue Oct 9 01:40:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "539d9689", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4285 (Tue Oct 9 01:40:14 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4285 (Tue Oct 9 01:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffdea53d", "local_conf_timestamp": 4285, "host-ts": 4285}, "global_maintenance": false} lago.ssh: DEBUG: start task:2ce66497-456f-4e57-a3f8-fcf8117a4263:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2ce66497-456f-4e57-a3f8-fcf8117a4263:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4d2c78c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4d2c78c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4d2c78c6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4297 (Tue Oct 9 01:40:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4297 (Tue Oct 9 01:40:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4722e013", "local_conf_timestamp": 4297, "host-ts": 4297}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4295 (Tue Oct 9 01:40:24 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4295 (Tue Oct 9 01:40:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fb985a38", "local_conf_timestamp": 4295, "host-ts": 4295}, "global_maintenance": false} lago.ssh: DEBUG: start task:2c93f2a0-8500-4528-a7ed-702eaef4866b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2c93f2a0-8500-4528-a7ed-702eaef4866b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4e3af0b2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4e3af0b2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4e3af0b2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4297 (Tue Oct 9 01:40:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4297 (Tue Oct 9 01:40:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4722e013", "local_conf_timestamp": 4297, "host-ts": 4297}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4295 (Tue Oct 9 01:40:24 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4295 (Tue Oct 9 01:40:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fb985a38", "local_conf_timestamp": 4295, "host-ts": 4295}, "global_maintenance": false} lago.ssh: DEBUG: start task:dba870a3-6461-4e57-8b68-fb5174d97dd7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dba870a3-6461-4e57-8b68-fb5174d97dd7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 554af398 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 554af398 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 554af398 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4307 (Tue Oct 9 01:40:35 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4307 (Tue Oct 9 01:40:36 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "3ab38b8d", "local_conf_timestamp": 4307, "host-ts": 4307}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4305 (Tue Oct 9 01:40:33 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4305 (Tue Oct 9 01:40:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9edda5b0", "local_conf_timestamp": 4305, "host-ts": 4305}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:e6633f02-b230-430f-9359-24fe2f4a7b4b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: cli: DEBUG: signal 15 was caught root: ERROR: end task:e6633f02-b230-430f-9359-24fe2f4a7b4b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 360, in get_ssh_client timeout=ssh_timeout, File "/usr/lib/python2.7/site-packages/paramiko/client.py", line 338, in connect t.start_client(timeout=timeout) File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 496, in start_client event.wait(0.1) File "/usr/lib64/python2.7/threading.py", line 622, in wait self.__cond.wait(timeout, balancing) File "/usr/lib64/python2.7/threading.py", line 362, in wait _sleep(delay) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 127, in local_maintenance ret = host.ssh(["hosted-engine", "--set-maintenance", "--mode=none"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 46, in ssh password=password, File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 360, in get_ssh_client timeout=ssh_timeout, File "/usr/lib/python2.7/site-packages/paramiko/client.py", line 338, in connect t.start_client(timeout=timeout) File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 496, in start_client event.wait(0.1) File "/usr/lib64/python2.7/threading.py", line 622, in wait self.__cond.wait(timeout, balancing) File "/usr/lib64/python2.7/threading.py", line 362, in wait _sleep(delay) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:3c73e9c4-b309-40fb-b1fa-ed36da927d07:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:3c73e9c4-b309-40fb-b1fa-ed36da927d07:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 24a15502 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 24a15502 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 24a15502 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Tue Oct 9 01:39:06 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4217 (Tue Oct 9 01:39:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "43ec0e80", "local_conf_timestamp": 4217, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Tue Oct 9 01:39:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4225 (Tue Oct 9 01:39:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1fcdbfe0-01de-40ae-bf85-7a17775da330:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1fcdbfe0-01de-40ae-bf85-7a17775da330:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 25aded20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 25aded20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:f3a614d4-1caf-4671-8ebc-2dfde68a8388:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f3a614d4-1caf-4671-8ebc-2dfde68a8388:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 26c47904 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 26c47904 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 26c47904 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Tue Oct 9 01:39:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4227 (Tue Oct 9 01:39:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fc34c880", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Tue Oct 9 01:39:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4225 (Tue Oct 9 01:39:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9fcf021e-6d26-468f-b607-a3670ee1eaaf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9fcf021e-6d26-468f-b607-a3670ee1eaaf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2e05fc88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2e05fc88 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2e05fc88 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Tue Oct 9 01:39:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4248 (Tue Oct 9 01:39:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67b39f04", "local_conf_timestamp": 4248, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Tue Oct 9 01:39:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4245 (Tue Oct 9 01:39:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7fd6798f", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f01bca9b-8543-4d58-939b-b3e24fff320b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f01bca9b-8543-4d58-939b-b3e24fff320b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 30f91402 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 30f91402 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 30f91402 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Tue Oct 9 01:39:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4248 (Tue Oct 9 01:39:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67b39f04", "local_conf_timestamp": 4248, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Tue Oct 9 01:39:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4245 (Tue Oct 9 01:39:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7fd6798f", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1cd91097-ce30-425f-856c-73f7581cc7bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1cd91097-ce30-425f-856c-73f7581cc7bf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 34323e5a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 34323e5a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 34323e5a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Tue Oct 9 01:39:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Tue Oct 9 01:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Tue Oct 9 01:39:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Tue Oct 9 01:39:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3aa0fe5c-5836-4e7b-9c7b-8aa0d676807d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3aa0fe5c-5836-4e7b-9c7b-8aa0d676807d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 371b6650 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 371b6650 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 371b6650 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Tue Oct 9 01:39:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Tue Oct 9 01:39:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Tue Oct 9 01:39:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Tue Oct 9 01:39:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cb267a07-39bb-4c78-aca2-d1d0652d6c7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cb267a07-39bb-4c78-aca2-d1d0652d6c7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3a5cb1b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3a5cb1b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3a5cb1b6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Tue Oct 9 01:39:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4267 (Tue Oct 9 01:39:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "701bf950", "local_conf_timestamp": 4267, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4265 (Tue Oct 9 01:39:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4265 (Tue Oct 9 01:39:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775b8785", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1da66f14-fc55-4fdf-a10f-bbd07b7dc01e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1da66f14-fc55-4fdf-a10f-bbd07b7dc01e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3d5da38e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3d5da38e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3d5da38e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Tue Oct 9 01:39:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4267 (Tue Oct 9 01:39:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "701bf950", "local_conf_timestamp": 4267, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4265 (Tue Oct 9 01:39:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4265 (Tue Oct 9 01:39:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "775b8785", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:489d8ae3-b227-4acc-8a31-9d0574f90918:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:489d8ae3-b227-4acc-8a31-9d0574f90918:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 40834a0a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 40834a0a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 40834a0a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Tue Oct 9 01:40:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4277 (Tue Oct 9 01:40:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4275 (Tue Oct 9 01:40:04 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4275 (Tue Oct 9 01:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:55a1f1ed-a80e-4e19-87b5-1e32bc97e4c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:55a1f1ed-a80e-4e19-87b5-1e32bc97e4c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 439d8a48 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 439d8a48 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 439d8a48 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Tue Oct 9 01:40:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4277 (Tue Oct 9 01:40:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4275 (Tue Oct 9 01:40:04 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4275 (Tue Oct 9 01:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7268f134-03c9-456b-b9a5-042f135adc89:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7268f134-03c9-456b-b9a5-042f135adc89:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 46bdf672 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 46bdf672 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 46bdf672 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Tue Oct 9 01:40:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4287 (Tue Oct 9 01:40:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "539d9689", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4285 (Tue Oct 9 01:40:14 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4285 (Tue Oct 9 01:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffdea53d", "local_conf_timestamp": 4285, "host-ts": 4285}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5b5ddba0-e48c-4425-b001-ee2a99288230:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5b5ddba0-e48c-4425-b001-ee2a99288230:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4a0f5500 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4a0f5500 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4a0f5500 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Tue Oct 9 01:40:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4287 (Tue Oct 9 01:40:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "539d9689", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4285 (Tue Oct 9 01:40:14 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4285 (Tue Oct 9 01:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffdea53d", "local_conf_timestamp": 4285, "host-ts": 4285}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2ce66497-456f-4e57-a3f8-fcf8117a4263:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2ce66497-456f-4e57-a3f8-fcf8117a4263:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4d2c78c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4d2c78c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4d2c78c6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4297 (Tue Oct 9 01:40:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4297 (Tue Oct 9 01:40:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4722e013", "local_conf_timestamp": 4297, "host-ts": 4297}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4295 (Tue Oct 9 01:40:24 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4295 (Tue Oct 9 01:40:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fb985a38", "local_conf_timestamp": 4295, "host-ts": 4295}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2c93f2a0-8500-4528-a7ed-702eaef4866b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2c93f2a0-8500-4528-a7ed-702eaef4866b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4e3af0b2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4e3af0b2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4e3af0b2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4297 (Tue Oct 9 01:40:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4297 (Tue Oct 9 01:40:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4722e013", "local_conf_timestamp": 4297, "host-ts": 4297}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4295 (Tue Oct 9 01:40:24 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4295 (Tue Oct 9 01:40:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fb985a38", "local_conf_timestamp": 4295, "host-ts": 4295}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dba870a3-6461-4e57-8b68-fb5174d97dd7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dba870a3-6461-4e57-8b68-fb5174d97dd7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 554af398 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 554af398 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 554af398 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4307 (Tue Oct 9 01:40:35 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4307 (Tue Oct 9 01:40:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "3ab38b8d", "local_conf_timestamp": 4307, "host-ts": 4307}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4305 (Tue Oct 9 01:40:33 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4305 (Tue Oct 9 01:40:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9edda5b0", "local_conf_timestamp": 4305, "host-ts": 4305}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:e6633f02-b230-430f-9359-24fe2f4a7b4b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\ncli: DEBUG: signal 15 was caught\nroot: ERROR: end task:e6633f02-b230-430f-9359-24fe2f4a7b4b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 360, in get_ssh_client\n timeout=ssh_timeout,\n File "/usr/lib/python2.7/site-packages/paramiko/client.py", line 338, in connect\n t.start_client(timeout=timeout)\n File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 496, in start_client\n event.wait(0.1)\n File "/usr/lib64/python2.7/threading.py", line 622, in wait\n self.__cond.wait(timeout, balancing)\n File "/usr/lib64/python2.7/threading.py", line 362, in wait\n _sleep(delay)\n File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler\n sys.exit(128 + signum)\n\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/385/ Build Number: 385 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.clear_global_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 76, in clear_global_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/386/ Build Number: 386 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:f40858d2-db02-4a55-8a1e-4d8142a10e9c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:f40858d2-db02-4a55-8a1e-4d8142a10e9c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running ebc9270a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ebc9270a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command ebc9270a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4170 (Tue Oct 9 17:43:56 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4171 (Tue Oct 9 17:43:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3097b0d9", "local_conf_timestamp": 4171, "host-ts": 4170}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4174 (Tue Oct 9 17:43:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4174 (Tue Oct 9 17:43:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "094a8b0e", "local_conf_timestamp": 4174, "host-ts": 4174}, "global_maintenance": false} lago.ssh: DEBUG: start task:2fc1546c-8d82-415a-8e04-4a8d9be86e47:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2fc1546c-8d82-415a-8e04-4a8d9be86e47:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ecd11536 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command ecd11536 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:74bf5d4e-94cc-4f22-960f-d7c3418fa82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:74bf5d4e-94cc-4f22-960f-d7c3418fa82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running eddc5c7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command eddc5c7e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command eddc5c7e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Tue Oct 9 17:44:06 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4181 (Tue Oct 9 17:44:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dbf5a4a2", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Tue Oct 9 17:44:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Tue Oct 9 17:44:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41aabeab", "local_conf_timestamp": 4185, "host-ts": 4184}, "global_maintenance": false} lago.ssh: DEBUG: start task:b7398a1a-3570-48ce-b17f-a576c81ce310:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b7398a1a-3570-48ce-b17f-a576c81ce310:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f51b7452 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f51b7452 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f51b7452 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Tue Oct 9 17:44:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4191 (Tue Oct 9 17:44:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24fdb3b1", "local_conf_timestamp": 4191, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Tue Oct 9 17:44:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Tue Oct 9 17:44:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41aabeab", "local_conf_timestamp": 4185, "host-ts": 4184}, "global_maintenance": false} lago.ssh: DEBUG: start task:96fd0ee0-8c27-4d74-b236-1ee63d4ad3ae:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:96fd0ee0-8c27-4d74-b236-1ee63d4ad3ae:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f862f194 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f862f194 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f862f194 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Tue Oct 9 17:44:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4202 (Tue Oct 9 17:44:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffb7d494", "local_conf_timestamp": 4202, "host-ts": 4202}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Tue Oct 9 17:44:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4195 (Tue Oct 9 17:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af3261be", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:c426afc9-ece6-448f-a4c1-e610003738f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c426afc9-ece6-448f-a4c1-e610003738f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fbd4490e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fbd4490e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fbd4490e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Tue Oct 9 17:44:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4202 (Tue Oct 9 17:44:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffb7d494", "local_conf_timestamp": 4202, "host-ts": 4202}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Tue Oct 9 17:44:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4195 (Tue Oct 9 17:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af3261be", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:e44172d5-3c56-4629-ae94-08e4ade9041b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e44172d5-3c56-4629-ae94-08e4ade9041b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fec45f50 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fec45f50 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fec45f50 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Tue Oct 9 17:44:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4212 (Tue Oct 9 17:44:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5184ea56", "local_conf_timestamp": 4212, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Tue Oct 9 17:44:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Tue Oct 9 17:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:6fb1e4a7-a410-48bb-b0a2-0e8ed5cd13db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6fb1e4a7-a410-48bb-b0a2-0e8ed5cd13db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 01a6f6ce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 01a6f6ce on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 01a6f6ce on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Tue Oct 9 17:44:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4212 (Tue Oct 9 17:44:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5184ea56", "local_conf_timestamp": 4212, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Tue Oct 9 17:44:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Tue Oct 9 17:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:243fc07f-957b-4c59-b1cc-3a58149df512:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:243fc07f-957b-4c59-b1cc-3a58149df512:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 04c022e0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 04c022e0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 04c022e0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Tue Oct 9 17:44:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4222 (Tue Oct 9 17:44:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Tue Oct 9 17:44:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4215 (Tue Oct 9 17:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:cf425d1c-f3fd-4e95-8e6c-71704714db5a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cf425d1c-f3fd-4e95-8e6c-71704714db5a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 07e2e5b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 07e2e5b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 07e2e5b6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Tue Oct 9 17:44:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4222 (Tue Oct 9 17:44:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Tue Oct 9 17:44:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4215 (Tue Oct 9 17:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:d396491b-c7aa-46d5-b180-ad71eb6667e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d396491b-c7aa-46d5-b180-ad71eb6667e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0ada0cea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0ada0cea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0ada0cea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Tue Oct 9 17:44:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4232 (Tue Oct 9 17:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d6939193", "local_conf_timestamp": 4232, "host-ts": 4232}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Tue Oct 9 17:44:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Tue Oct 9 17:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ff492a2b", "local_conf_timestamp": 4226, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:7740071a-82d7-48d1-85d4-f201087912e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7740071a-82d7-48d1-85d4-f201087912e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0e43d898 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0e43d898 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0e43d898 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Tue Oct 9 17:45:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4242 (Tue Oct 9 17:45:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ae8255f", "local_conf_timestamp": 4242, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Tue Oct 9 17:44:59 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4235 (Tue Oct 9 17:45:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:3c516494-fe14-4f0a-9ed1-ad305481fcf3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3c516494-fe14-4f0a-9ed1-ad305481fcf3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 127900be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 127900be on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 127900be on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Tue Oct 9 17:45:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4242 (Tue Oct 9 17:45:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ae8255f", "local_conf_timestamp": 4242, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Tue Oct 9 17:44:59 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4235 (Tue Oct 9 17:45:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:894b78f9-82c7-4274-a24b-1a9939bb7944:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:894b78f9-82c7-4274-a24b-1a9939bb7944:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1590e758 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1590e758 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1590e758 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4252 (Tue Oct 9 17:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4252 (Tue Oct 9 17:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "84db1b9d", "local_conf_timestamp": 4252, "host-ts": 4252}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Tue Oct 9 17:45:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4245 (Tue Oct 9 17:45:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:98f36712-d6c4-4cda-be0c-84b9245bfe33:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:98f36712-d6c4-4cda-be0c-84b9245bfe33:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 188e7eac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 188e7eac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 188e7eac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4252 (Tue Oct 9 17:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4252 (Tue Oct 9 17:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "84db1b9d", "local_conf_timestamp": 4252, "host-ts": 4252}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Tue Oct 9 17:45:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4245 (Tue Oct 9 17:45:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:60489184-3476-4ae0-bc53-b2fd9b55a327:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:60489184-3476-4ae0-bc53-b2fd9b55a327:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1b6f6cd0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1b6f6cd0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1b6f6cd0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4262 (Tue Oct 9 17:45:28 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4263 (Tue Oct 9 17:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "913c9615", "local_conf_timestamp": 4263, "host-ts": 4262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Tue Oct 9 17:45:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Tue Oct 9 17:45:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} lago.ssh: DEBUG: start task:e873afd6-b192-4f55-aef9-60e067b560ad:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e873afd6-b192-4f55-aef9-60e067b560ad:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1c83dd36 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1c83dd36 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1c83dd36 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4262 (Tue Oct 9 17:45:28 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4263 (Tue Oct 9 17:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "913c9615", "local_conf_timestamp": 4263, "host-ts": 4262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Tue Oct 9 17:45:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Tue Oct 9 17:45:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:f40858d2-db02-4a55-8a1e-4d8142a10e9c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:f40858d2-db02-4a55-8a1e-4d8142a10e9c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running ebc9270a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ebc9270a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command ebc9270a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4170 (Tue Oct 9 17:43:56 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4171 (Tue Oct 9 17:43:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3097b0d9", "local_conf_timestamp": 4171, "host-ts": 4170}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4174 (Tue Oct 9 17:43:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4174 (Tue Oct 9 17:43:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "094a8b0e", "local_conf_timestamp": 4174, "host-ts": 4174}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2fc1546c-8d82-415a-8e04-4a8d9be86e47:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2fc1546c-8d82-415a-8e04-4a8d9be86e47:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ecd11536 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command ecd11536 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:74bf5d4e-94cc-4f22-960f-d7c3418fa82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:74bf5d4e-94cc-4f22-960f-d7c3418fa82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running eddc5c7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command eddc5c7e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command eddc5c7e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Tue Oct 9 17:44:06 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4181 (Tue Oct 9 17:44:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dbf5a4a2", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Tue Oct 9 17:44:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Tue Oct 9 17:44:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41aabeab", "local_conf_timestamp": 4185, "host-ts": 4184}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b7398a1a-3570-48ce-b17f-a576c81ce310:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b7398a1a-3570-48ce-b17f-a576c81ce310:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f51b7452 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f51b7452 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f51b7452 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Tue Oct 9 17:44:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4191 (Tue Oct 9 17:44:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24fdb3b1", "local_conf_timestamp": 4191, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Tue Oct 9 17:44:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Tue Oct 9 17:44:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "41aabeab", "local_conf_timestamp": 4185, "host-ts": 4184}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:96fd0ee0-8c27-4d74-b236-1ee63d4ad3ae:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:96fd0ee0-8c27-4d74-b236-1ee63d4ad3ae:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f862f194 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f862f194 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f862f194 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Tue Oct 9 17:44:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4202 (Tue Oct 9 17:44:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffb7d494", "local_conf_timestamp": 4202, "host-ts": 4202}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Tue Oct 9 17:44:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4195 (Tue Oct 9 17:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af3261be", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c426afc9-ece6-448f-a4c1-e610003738f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c426afc9-ece6-448f-a4c1-e610003738f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fbd4490e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fbd4490e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fbd4490e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Tue Oct 9 17:44:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4202 (Tue Oct 9 17:44:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ffb7d494", "local_conf_timestamp": 4202, "host-ts": 4202}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Tue Oct 9 17:44:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4195 (Tue Oct 9 17:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "af3261be", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e44172d5-3c56-4629-ae94-08e4ade9041b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e44172d5-3c56-4629-ae94-08e4ade9041b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fec45f50 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fec45f50 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fec45f50 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Tue Oct 9 17:44:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4212 (Tue Oct 9 17:44:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5184ea56", "local_conf_timestamp": 4212, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Tue Oct 9 17:44:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Tue Oct 9 17:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6fb1e4a7-a410-48bb-b0a2-0e8ed5cd13db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6fb1e4a7-a410-48bb-b0a2-0e8ed5cd13db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 01a6f6ce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 01a6f6ce on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 01a6f6ce on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Tue Oct 9 17:44:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4212 (Tue Oct 9 17:44:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5184ea56", "local_conf_timestamp": 4212, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Tue Oct 9 17:44:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Tue Oct 9 17:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:243fc07f-957b-4c59-b1cc-3a58149df512:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:243fc07f-957b-4c59-b1cc-3a58149df512:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 04c022e0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 04c022e0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 04c022e0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Tue Oct 9 17:44:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4222 (Tue Oct 9 17:44:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Tue Oct 9 17:44:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4215 (Tue Oct 9 17:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cf425d1c-f3fd-4e95-8e6c-71704714db5a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cf425d1c-f3fd-4e95-8e6c-71704714db5a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 07e2e5b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 07e2e5b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 07e2e5b6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Tue Oct 9 17:44:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4222 (Tue Oct 9 17:44:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Tue Oct 9 17:44:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4215 (Tue Oct 9 17:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d396491b-c7aa-46d5-b180-ad71eb6667e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d396491b-c7aa-46d5-b180-ad71eb6667e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0ada0cea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0ada0cea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0ada0cea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Tue Oct 9 17:44:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4232 (Tue Oct 9 17:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d6939193", "local_conf_timestamp": 4232, "host-ts": 4232}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Tue Oct 9 17:44:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Tue Oct 9 17:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ff492a2b", "local_conf_timestamp": 4226, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7740071a-82d7-48d1-85d4-f201087912e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7740071a-82d7-48d1-85d4-f201087912e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0e43d898 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0e43d898 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0e43d898 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Tue Oct 9 17:45:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4242 (Tue Oct 9 17:45:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ae8255f", "local_conf_timestamp": 4242, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Tue Oct 9 17:44:59 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4235 (Tue Oct 9 17:45:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3c516494-fe14-4f0a-9ed1-ad305481fcf3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3c516494-fe14-4f0a-9ed1-ad305481fcf3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 127900be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 127900be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 127900be on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Tue Oct 9 17:45:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4242 (Tue Oct 9 17:45:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ae8255f", "local_conf_timestamp": 4242, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Tue Oct 9 17:44:59 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4235 (Tue Oct 9 17:45:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:894b78f9-82c7-4274-a24b-1a9939bb7944:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:894b78f9-82c7-4274-a24b-1a9939bb7944:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1590e758 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1590e758 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1590e758 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4252 (Tue Oct 9 17:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4252 (Tue Oct 9 17:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "84db1b9d", "local_conf_timestamp": 4252, "host-ts": 4252}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Tue Oct 9 17:45:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4245 (Tue Oct 9 17:45:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:98f36712-d6c4-4cda-be0c-84b9245bfe33:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:98f36712-d6c4-4cda-be0c-84b9245bfe33:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 188e7eac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 188e7eac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 188e7eac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4252 (Tue Oct 9 17:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4252 (Tue Oct 9 17:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "84db1b9d", "local_conf_timestamp": 4252, "host-ts": 4252}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Tue Oct 9 17:45:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4245 (Tue Oct 9 17:45:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:60489184-3476-4ae0-bc53-b2fd9b55a327:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:60489184-3476-4ae0-bc53-b2fd9b55a327:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1b6f6cd0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1b6f6cd0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1b6f6cd0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4262 (Tue Oct 9 17:45:28 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4263 (Tue Oct 9 17:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "913c9615", "local_conf_timestamp": 4263, "host-ts": 4262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Tue Oct 9 17:45:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Tue Oct 9 17:45:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e873afd6-b192-4f55-aef9-60e067b560ad:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e873afd6-b192-4f55-aef9-60e067b560ad:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1c83dd36 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1c83dd36 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1c83dd36 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4262 (Tue Oct 9 17:45:28 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4263 (Tue Oct 9 17:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "913c9615", "local_conf_timestamp": 4263, "host-ts": 4262}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Tue Oct 9 17:45:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Tue Oct 9 17:45:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/387/ Build Number: 387 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:2795dfd4-133a-47d0-bf71-d1be012c19ed:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:2795dfd4-133a-47d0-bf71-d1be012c19ed:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running f555bcc8 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f555bcc8 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command f555bcc8 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4147 (Wed Oct 10 01:43:43 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4147 (Wed Oct 10 01:43:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "584b9a07", "local_conf_timestamp": 4147, "host-ts": 4147}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4156 (Wed Oct 10 01:43:52 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4156 (Wed Oct 10 01:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false} lago.ssh: DEBUG: start task:81db9307-a783-43ee-9758-fb88c84b73b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:81db9307-a783-43ee-9758-fb88c84b73b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f66009d4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command f66009d4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:3acfc4d4-b4a8-4268-976c-f6beb706d1a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3acfc4d4-b4a8-4268-976c-f6beb706d1a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f78418e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f78418e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f78418e6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4157 (Wed Oct 10 01:43:54 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4157 (Wed Oct 10 01:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "842cda38", "local_conf_timestamp": 4157, "host-ts": 4157}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4156 (Wed Oct 10 01:43:52 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4156 (Wed Oct 10 01:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false} lago.ssh: DEBUG: start task:dc9c118b-918d-453a-92ea-93f92f1ff4b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dc9c118b-918d-453a-92ea-93f92f1ff4b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running feb0545e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command feb0545e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command feb0545e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Wed Oct 10 01:44:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4178 (Wed Oct 10 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Wed Oct 10 01:44:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4177 (Wed Oct 10 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9e8e9cd5", "local_conf_timestamp": 4177, "host-ts": 4176}, "global_maintenance": false} lago.ssh: DEBUG: start task:1cf6b1c4-96f5-425d-a64b-98fb7eaddb76:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1cf6b1c4-96f5-425d-a64b-98fb7eaddb76:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0242104e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0242104e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0242104e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Wed Oct 10 01:44:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4178 (Wed Oct 10 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Wed Oct 10 01:44:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4177 (Wed Oct 10 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9e8e9cd5", "local_conf_timestamp": 4177, "host-ts": 4176}, "global_maintenance": false} lago.ssh: DEBUG: start task:8c7d4176-780d-4624-b5a0-89a8b8e0e391:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8c7d4176-780d-4624-b5a0-89a8b8e0e391:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0590dce4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0590dce4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0590dce4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Wed Oct 10 01:44:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Wed Oct 10 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4186 (Wed Oct 10 01:44:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4186 (Wed Oct 10 01:44:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false} lago.ssh: DEBUG: start task:1376ab24-b7e7-4640-a789-b918bf76bcb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1376ab24-b7e7-4640-a789-b918bf76bcb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 088cbd78 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 088cbd78 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 088cbd78 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Wed Oct 10 01:44:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Wed Oct 10 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4186 (Wed Oct 10 01:44:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4186 (Wed Oct 10 01:44:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false} lago.ssh: DEBUG: start task:03697e30-6d07-4d43-8ceb-2276e06c94e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:03697e30-6d07-4d43-8ceb-2276e06c94e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0b810f2a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0b810f2a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0b810f2a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Wed Oct 10 01:44:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Wed Oct 10 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Wed Oct 10 01:44:32 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Wed Oct 10 01:44:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} lago.ssh: DEBUG: start task:3ef90b1f-3611-4b8d-8238-fa767058e82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3ef90b1f-3611-4b8d-8238-fa767058e82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0e91c86c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0e91c86c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0e91c86c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Wed Oct 10 01:44:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Wed Oct 10 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Wed Oct 10 01:44:32 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Wed Oct 10 01:44:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} lago.ssh: DEBUG: start task:09dc2dfd-fd41-43c3-8cb9-7016dad06498:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:09dc2dfd-fd41-43c3-8cb9-7016dad06498:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 11a52580 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 11a52580 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 11a52580 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Wed Oct 10 01:44:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4208 (Wed Oct 10 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Wed Oct 10 01:44:43 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4206 (Wed Oct 10 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:e9ff148b-4346-40d6-bff5-d0e4c87a81d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e9ff148b-4346-40d6-bff5-d0e4c87a81d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1485bba2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1485bba2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1485bba2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Wed Oct 10 01:44:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4208 (Wed Oct 10 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Wed Oct 10 01:44:43 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4206 (Wed Oct 10 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:22aba53c-c592-4f53-a944-0254cd90191c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:22aba53c-c592-4f53-a944-0254cd90191c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 180ee334 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 180ee334 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 180ee334 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Wed Oct 10 01:44:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Wed Oct 10 01:44:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Wed Oct 10 01:44:53 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4217 (Wed Oct 10 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3b2a54d", "local_conf_timestamp": 4217, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:6c731747-bdba-4301-8eef-0dc1c1d5bfb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6c731747-bdba-4301-8eef-0dc1c1d5bfb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1b0f32e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1b0f32e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1b0f32e6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Wed Oct 10 01:44:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Wed Oct 10 01:44:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Wed Oct 10 01:44:53 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4217 (Wed Oct 10 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3b2a54d", "local_conf_timestamp": 4217, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:373f1b5f-5bc1-4752-8264-e3f1dca21236:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:373f1b5f-5bc1-4752-8264-e3f1dca21236:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1dfa20c4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1dfa20c4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1dfa20c4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Wed Oct 10 01:45:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Wed Oct 10 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8043cb00", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Wed Oct 10 01:45:02 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4226 (Wed Oct 10 01:45:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:85f3fce6-2b40-4f46-8277-ac12256b8e77:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:85f3fce6-2b40-4f46-8277-ac12256b8e77:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1f05efac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1f05efac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1f05efac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Wed Oct 10 01:45:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Wed Oct 10 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8043cb00", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Wed Oct 10 01:45:02 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4226 (Wed Oct 10 01:45:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:b588e6f4-8366-426f-ab67-394dc7635d2a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b588e6f4-8366-426f-ab67-394dc7635d2a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 26058bc8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 26058bc8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 26058bc8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Wed Oct 10 01:45:14 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4238 (Wed Oct 10 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "89ecf2e7", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Wed Oct 10 01:45:12 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4236 (Wed Oct 10 01:45:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f183ffe", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:2795dfd4-133a-47d0-bf71-d1be012c19ed:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:2795dfd4-133a-47d0-bf71-d1be012c19ed:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running f555bcc8 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f555bcc8 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command f555bcc8 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4147 (Wed Oct 10 01:43:43 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4147 (Wed Oct 10 01:43:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "584b9a07", "local_conf_timestamp": 4147, "host-ts": 4147}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4156 (Wed Oct 10 01:43:52 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4156 (Wed Oct 10 01:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:81db9307-a783-43ee-9758-fb88c84b73b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:81db9307-a783-43ee-9758-fb88c84b73b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f66009d4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command f66009d4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:3acfc4d4-b4a8-4268-976c-f6beb706d1a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3acfc4d4-b4a8-4268-976c-f6beb706d1a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f78418e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f78418e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f78418e6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4157 (Wed Oct 10 01:43:54 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4157 (Wed Oct 10 01:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "842cda38", "local_conf_timestamp": 4157, "host-ts": 4157}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4156 (Wed Oct 10 01:43:52 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4156 (Wed Oct 10 01:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dc9c118b-918d-453a-92ea-93f92f1ff4b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dc9c118b-918d-453a-92ea-93f92f1ff4b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running feb0545e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command feb0545e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command feb0545e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Wed Oct 10 01:44:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4178 (Wed Oct 10 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Wed Oct 10 01:44:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4177 (Wed Oct 10 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9e8e9cd5", "local_conf_timestamp": 4177, "host-ts": 4176}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1cf6b1c4-96f5-425d-a64b-98fb7eaddb76:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1cf6b1c4-96f5-425d-a64b-98fb7eaddb76:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0242104e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0242104e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0242104e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Wed Oct 10 01:44:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4178 (Wed Oct 10 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Wed Oct 10 01:44:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4177 (Wed Oct 10 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9e8e9cd5", "local_conf_timestamp": 4177, "host-ts": 4176}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8c7d4176-780d-4624-b5a0-89a8b8e0e391:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8c7d4176-780d-4624-b5a0-89a8b8e0e391:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0590dce4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0590dce4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0590dce4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Wed Oct 10 01:44:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Wed Oct 10 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4186 (Wed Oct 10 01:44:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4186 (Wed Oct 10 01:44:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1376ab24-b7e7-4640-a789-b918bf76bcb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1376ab24-b7e7-4640-a789-b918bf76bcb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 088cbd78 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 088cbd78 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 088cbd78 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Wed Oct 10 01:44:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Wed Oct 10 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4186 (Wed Oct 10 01:44:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4186 (Wed Oct 10 01:44:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:03697e30-6d07-4d43-8ceb-2276e06c94e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:03697e30-6d07-4d43-8ceb-2276e06c94e0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0b810f2a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0b810f2a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0b810f2a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Wed Oct 10 01:44:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Wed Oct 10 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Wed Oct 10 01:44:32 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Wed Oct 10 01:44:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3ef90b1f-3611-4b8d-8238-fa767058e82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3ef90b1f-3611-4b8d-8238-fa767058e82a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0e91c86c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0e91c86c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0e91c86c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Wed Oct 10 01:44:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Wed Oct 10 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a9b8c724", "local_conf_timestamp": 4199, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Wed Oct 10 01:44:32 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Wed Oct 10 01:44:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:09dc2dfd-fd41-43c3-8cb9-7016dad06498:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:09dc2dfd-fd41-43c3-8cb9-7016dad06498:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 11a52580 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 11a52580 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 11a52580 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Wed Oct 10 01:44:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4208 (Wed Oct 10 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Wed Oct 10 01:44:43 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4206 (Wed Oct 10 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e9ff148b-4346-40d6-bff5-d0e4c87a81d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e9ff148b-4346-40d6-bff5-d0e4c87a81d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1485bba2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1485bba2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1485bba2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Wed Oct 10 01:44:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4208 (Wed Oct 10 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9f0f9d41", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Wed Oct 10 01:44:43 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4206 (Wed Oct 10 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:22aba53c-c592-4f53-a944-0254cd90191c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:22aba53c-c592-4f53-a944-0254cd90191c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 180ee334 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 180ee334 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 180ee334 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Wed Oct 10 01:44:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Wed Oct 10 01:44:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Wed Oct 10 01:44:53 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4217 (Wed Oct 10 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3b2a54d", "local_conf_timestamp": 4217, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6c731747-bdba-4301-8eef-0dc1c1d5bfb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6c731747-bdba-4301-8eef-0dc1c1d5bfb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1b0f32e6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1b0f32e6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1b0f32e6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Wed Oct 10 01:44:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Wed Oct 10 01:44:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Wed Oct 10 01:44:53 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4217 (Wed Oct 10 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3b2a54d", "local_conf_timestamp": 4217, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:373f1b5f-5bc1-4752-8264-e3f1dca21236:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:373f1b5f-5bc1-4752-8264-e3f1dca21236:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1dfa20c4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1dfa20c4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1dfa20c4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Wed Oct 10 01:45:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Wed Oct 10 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8043cb00", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Wed Oct 10 01:45:02 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4226 (Wed Oct 10 01:45:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:85f3fce6-2b40-4f46-8277-ac12256b8e77:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:85f3fce6-2b40-4f46-8277-ac12256b8e77:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1f05efac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1f05efac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1f05efac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Wed Oct 10 01:45:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Wed Oct 10 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8043cb00", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Wed Oct 10 01:45:02 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4226 (Wed Oct 10 01:45:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b588e6f4-8366-426f-ab67-394dc7635d2a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b588e6f4-8366-426f-ab67-394dc7635d2a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 26058bc8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 26058bc8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 26058bc8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Wed Oct 10 01:45:14 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4238 (Wed Oct 10 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "89ecf2e7", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Wed Oct 10 01:45:12 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4236 (Wed Oct 10 01:45:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f183ffe", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/388/ Build Number: 388 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/389/ Build Number: 389 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... root: INFO: * Performing Activation... root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 79, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\nroot: INFO: * Performing Activation...\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/390/ Build Number: 390 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:d1b99ddc-5eec-4a54-87b8-95bea7c7e136:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:d1b99ddc-5eec-4a54-87b8-95bea7c7e136:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 0a093d06 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0a093d06 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 0a093d06 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4139 (Thu Oct 11 01:43:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4140 (Thu Oct 11 01:43:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d07c0cb", "local_conf_timestamp": 4140, "host-ts": 4139}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4150 (Thu Oct 11 01:43:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4150 (Thu Oct 11 01:43:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b303ddf", "local_conf_timestamp": 4150, "host-ts": 4150}, "global_maintenance": false} lago.ssh: DEBUG: start task:bce3d7ac-6586-4ef7-b831-1062b75de736:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bce3d7ac-6586-4ef7-b831-1062b75de736:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0b11e25c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 0b11e25c on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:9cf75bba-2ceb-4557-ae04-718af8118bb6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9cf75bba-2ceb-4557-ae04-718af8118bb6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0c3313c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0c3313c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0c3313c2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4149 (Thu Oct 11 01:43:23 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4149 (Thu Oct 11 01:43:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c33d24d6", "local_conf_timestamp": 4149, "host-ts": 4149}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4150 (Thu Oct 11 01:43:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4150 (Thu Oct 11 01:43:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b303ddf", "local_conf_timestamp": 4150, "host-ts": 4150}, "global_maintenance": false} lago.ssh: DEBUG: start task:93e69af4-3e74-445f-b6a4-8b080fa6a03b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:93e69af4-3e74-445f-b6a4-8b080fa6a03b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 14703d26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 14703d26 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 14703d26 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4159 (Thu Oct 11 01:43:33 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4159 (Thu Oct 11 01:43:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3c3533c5", "local_conf_timestamp": 4159, "host-ts": 4159}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4150 (Thu Oct 11 01:43:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4150 (Thu Oct 11 01:43:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b303ddf", "local_conf_timestamp": 4150, "host-ts": 4150}, "global_maintenance": false} lago.ssh: DEBUG: start task:aeef70b4-dd1c-4b0c-9354-d8c507c452ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aeef70b4-dd1c-4b0c-9354-d8c507c452ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 182d81d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 182d81d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 182d81d0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Thu Oct 11 01:43:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4170 (Thu Oct 11 01:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d199a7a4", "local_conf_timestamp": 4170, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4160 (Thu Oct 11 01:43:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4160 (Thu Oct 11 01:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "17fb3cd0", "local_conf_timestamp": 4160, "host-ts": 4160}, "global_maintenance": false} lago.ssh: DEBUG: start task:8920a229-ff66-4df4-80cb-adbcf9edd6e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8920a229-ff66-4df4-80cb-adbcf9edd6e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1b3d6138 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1b3d6138 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1b3d6138 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Thu Oct 11 01:43:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4170 (Thu Oct 11 01:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d199a7a4", "local_conf_timestamp": 4170, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4160 (Thu Oct 11 01:43:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4160 (Thu Oct 11 01:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "17fb3cd0", "local_conf_timestamp": 4160, "host-ts": 4160}, "global_maintenance": false} lago.ssh: DEBUG: start task:5fecb6a5-2f33-4ede-a85c-e4ce54521b5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5fecb6a5-2f33-4ede-a85c-e4ce54521b5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1e2e5f00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1e2e5f00 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1e2e5f00 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Thu Oct 11 01:43:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4179 (Thu Oct 11 01:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1f6d1083", "local_conf_timestamp": 4179, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4170 (Thu Oct 11 01:43:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4170 (Thu Oct 11 01:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6480f64d", "local_conf_timestamp": 4170, "host-ts": 4170}, "global_maintenance": false} lago.ssh: DEBUG: start task:c6e6802e-7c02-476b-9a83-8533aaf80a93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c6e6802e-7c02-476b-9a83-8533aaf80a93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 21fba7f0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 21fba7f0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 21fba7f0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Thu Oct 11 01:44:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4189 (Thu Oct 11 01:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Thu Oct 11 01:43:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4180 (Thu Oct 11 01:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5b67f37e", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false} lago.ssh: DEBUG: start task:9ba6756f-8ca8-4b00-a1c0-bfe8857fa1d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9ba6756f-8ca8-4b00-a1c0-bfe8857fa1d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 251c0150 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 251c0150 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 251c0150 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Thu Oct 11 01:44:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4189 (Thu Oct 11 01:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Thu Oct 11 01:43:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4180 (Thu Oct 11 01:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5b67f37e", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false} lago.ssh: DEBUG: start task:5c2077e4-e407-46bc-bb23-5b19c8de89d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5c2077e4-e407-46bc-bb23-5b19c8de89d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 28129248 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 28129248 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 28129248 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Thu Oct 11 01:44:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Thu Oct 11 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Thu Oct 11 01:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Thu Oct 11 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "28263ced", "local_conf_timestamp": 4191, "host-ts": 4190}, "global_maintenance": false} lago.ssh: DEBUG: start task:88de5a4d-ccd0-4be7-8b59-acb09cc9702a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:88de5a4d-ccd0-4be7-8b59-acb09cc9702a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2af8d2e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2af8d2e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2af8d2e2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Thu Oct 11 01:44:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4199 (Thu Oct 11 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Thu Oct 11 01:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Thu Oct 11 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "28263ced", "local_conf_timestamp": 4191, "host-ts": 4190}, "global_maintenance": false} lago.ssh: DEBUG: start task:6508ec9d-800e-4147-ac0a-473336106da4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6508ec9d-800e-4147-ac0a-473336106da4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2df4df86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2df4df86 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2df4df86 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Thu Oct 11 01:44:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Thu Oct 11 01:44:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Thu Oct 11 01:44:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4201 (Thu Oct 11 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:dc07fda0-c61f-4998-bfc8-9523b3412450:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dc07fda0-c61f-4998-bfc8-9523b3412450:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 30fc6d34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 30fc6d34 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 30fc6d34 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Thu Oct 11 01:44:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Thu Oct 11 01:44:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Thu Oct 11 01:44:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4201 (Thu Oct 11 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:553558cb-51aa-4aef-b7dd-6c20e89acacf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:553558cb-51aa-4aef-b7dd-6c20e89acacf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 34341d26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 34341d26 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 34341d26 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Thu Oct 11 01:44:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4220 (Thu Oct 11 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8199c00c", "local_conf_timestamp": 4220, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Thu Oct 11 01:44:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4211 (Thu Oct 11 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:3fe20750-3036-4ab5-bf9e-c7ad23067feb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3fe20750-3036-4ab5-bf9e-c7ad23067feb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 37c5d84e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 37c5d84e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 37c5d84e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Thu Oct 11 01:44:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4220 (Thu Oct 11 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8199c00c", "local_conf_timestamp": 4220, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Thu Oct 11 01:44:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4211 (Thu Oct 11 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:0dc81015-4239-4ac3-bb81-219fc3f1290e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0dc81015-4239-4ac3-bb81-219fc3f1290e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3aa2f010 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3aa2f010 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3aa2f010 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Thu Oct 11 01:44:44 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4230 (Thu Oct 11 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "13a582df", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Thu Oct 11 01:44:35 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4221 (Thu Oct 11 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cf93ebc4", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:be1e76bb-a2bf-4e81-94e0-afda3c6653bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:be1e76bb-a2bf-4e81-94e0-afda3c6653bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3bbfe1ce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3bbfe1ce on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3bbfe1ce on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Thu Oct 11 01:44:44 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4230 (Thu Oct 11 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "13a582df", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Thu Oct 11 01:44:35 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4221 (Thu Oct 11 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cf93ebc4", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:f5152f63-9a43-4490-a345-c76acb58b6be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f5152f63-9a43-4490-a345-c76acb58b6be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 42e06cbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 42e06cbc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 42e06cbc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Thu Oct 11 01:44:55 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4241 (Thu Oct 11 01:44:55 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "65421165", "local_conf_timestamp": 4241, "host-ts": 4240}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Thu Oct 11 01:44:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4232 (Thu Oct 11 01:44:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "37bcf585", "local_conf_timestamp": 4232, "host-ts": 4232}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:d1b99ddc-5eec-4a54-87b8-95bea7c7e136:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:d1b99ddc-5eec-4a54-87b8-95bea7c7e136:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 0a093d06 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0a093d06 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 0a093d06 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4139 (Thu Oct 11 01:43:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4140 (Thu Oct 11 01:43:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d07c0cb", "local_conf_timestamp": 4140, "host-ts": 4139}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4150 (Thu Oct 11 01:43:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4150 (Thu Oct 11 01:43:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b303ddf", "local_conf_timestamp": 4150, "host-ts": 4150}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bce3d7ac-6586-4ef7-b831-1062b75de736:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bce3d7ac-6586-4ef7-b831-1062b75de736:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0b11e25c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 0b11e25c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:9cf75bba-2ceb-4557-ae04-718af8118bb6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9cf75bba-2ceb-4557-ae04-718af8118bb6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0c3313c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0c3313c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0c3313c2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4149 (Thu Oct 11 01:43:23 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4149 (Thu Oct 11 01:43:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c33d24d6", "local_conf_timestamp": 4149, "host-ts": 4149}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4150 (Thu Oct 11 01:43:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4150 (Thu Oct 11 01:43:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b303ddf", "local_conf_timestamp": 4150, "host-ts": 4150}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:93e69af4-3e74-445f-b6a4-8b080fa6a03b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:93e69af4-3e74-445f-b6a4-8b080fa6a03b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 14703d26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 14703d26 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 14703d26 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4159 (Thu Oct 11 01:43:33 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4159 (Thu Oct 11 01:43:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3c3533c5", "local_conf_timestamp": 4159, "host-ts": 4159}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4150 (Thu Oct 11 01:43:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4150 (Thu Oct 11 01:43:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b303ddf", "local_conf_timestamp": 4150, "host-ts": 4150}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aeef70b4-dd1c-4b0c-9354-d8c507c452ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aeef70b4-dd1c-4b0c-9354-d8c507c452ef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 182d81d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 182d81d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 182d81d0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Thu Oct 11 01:43:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4170 (Thu Oct 11 01:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d199a7a4", "local_conf_timestamp": 4170, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4160 (Thu Oct 11 01:43:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4160 (Thu Oct 11 01:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "17fb3cd0", "local_conf_timestamp": 4160, "host-ts": 4160}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8920a229-ff66-4df4-80cb-adbcf9edd6e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8920a229-ff66-4df4-80cb-adbcf9edd6e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1b3d6138 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1b3d6138 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1b3d6138 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Thu Oct 11 01:43:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4170 (Thu Oct 11 01:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d199a7a4", "local_conf_timestamp": 4170, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4160 (Thu Oct 11 01:43:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4160 (Thu Oct 11 01:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "17fb3cd0", "local_conf_timestamp": 4160, "host-ts": 4160}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5fecb6a5-2f33-4ede-a85c-e4ce54521b5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5fecb6a5-2f33-4ede-a85c-e4ce54521b5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1e2e5f00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1e2e5f00 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1e2e5f00 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Thu Oct 11 01:43:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4179 (Thu Oct 11 01:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1f6d1083", "local_conf_timestamp": 4179, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4170 (Thu Oct 11 01:43:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4170 (Thu Oct 11 01:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6480f64d", "local_conf_timestamp": 4170, "host-ts": 4170}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c6e6802e-7c02-476b-9a83-8533aaf80a93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c6e6802e-7c02-476b-9a83-8533aaf80a93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 21fba7f0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 21fba7f0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 21fba7f0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Thu Oct 11 01:44:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4189 (Thu Oct 11 01:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Thu Oct 11 01:43:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4180 (Thu Oct 11 01:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5b67f37e", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9ba6756f-8ca8-4b00-a1c0-bfe8857fa1d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9ba6756f-8ca8-4b00-a1c0-bfe8857fa1d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 251c0150 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 251c0150 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 251c0150 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Thu Oct 11 01:44:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4189 (Thu Oct 11 01:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "92d84198", "local_conf_timestamp": 4189, "host-ts": 4189}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Thu Oct 11 01:43:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4180 (Thu Oct 11 01:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5b67f37e", "local_conf_timestamp": 4180, "host-ts": 4180}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5c2077e4-e407-46bc-bb23-5b19c8de89d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5c2077e4-e407-46bc-bb23-5b19c8de89d3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 28129248 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 28129248 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 28129248 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Thu Oct 11 01:44:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Thu Oct 11 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Thu Oct 11 01:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Thu Oct 11 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "28263ced", "local_conf_timestamp": 4191, "host-ts": 4190}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:88de5a4d-ccd0-4be7-8b59-acb09cc9702a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:88de5a4d-ccd0-4be7-8b59-acb09cc9702a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2af8d2e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2af8d2e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2af8d2e2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Thu Oct 11 01:44:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4199 (Thu Oct 11 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ceb7f5a", "local_conf_timestamp": 4199, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Thu Oct 11 01:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Thu Oct 11 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "28263ced", "local_conf_timestamp": 4191, "host-ts": 4190}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6508ec9d-800e-4147-ac0a-473336106da4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6508ec9d-800e-4147-ac0a-473336106da4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2df4df86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2df4df86 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2df4df86 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Thu Oct 11 01:44:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Thu Oct 11 01:44:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Thu Oct 11 01:44:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4201 (Thu Oct 11 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dc07fda0-c61f-4998-bfc8-9523b3412450:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dc07fda0-c61f-4998-bfc8-9523b3412450:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 30fc6d34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 30fc6d34 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 30fc6d34 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Thu Oct 11 01:44:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Thu Oct 11 01:44:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Thu Oct 11 01:44:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4201 (Thu Oct 11 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c71e15ce", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:553558cb-51aa-4aef-b7dd-6c20e89acacf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:553558cb-51aa-4aef-b7dd-6c20e89acacf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 34341d26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 34341d26 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 34341d26 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Thu Oct 11 01:44:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4220 (Thu Oct 11 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8199c00c", "local_conf_timestamp": 4220, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Thu Oct 11 01:44:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4211 (Thu Oct 11 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3fe20750-3036-4ab5-bf9e-c7ad23067feb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3fe20750-3036-4ab5-bf9e-c7ad23067feb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 37c5d84e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 37c5d84e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 37c5d84e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Thu Oct 11 01:44:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4220 (Thu Oct 11 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8199c00c", "local_conf_timestamp": 4220, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Thu Oct 11 01:44:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4211 (Thu Oct 11 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c358eacb", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0dc81015-4239-4ac3-bb81-219fc3f1290e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0dc81015-4239-4ac3-bb81-219fc3f1290e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3aa2f010 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3aa2f010 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3aa2f010 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Thu Oct 11 01:44:44 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4230 (Thu Oct 11 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "13a582df", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Thu Oct 11 01:44:35 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4221 (Thu Oct 11 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cf93ebc4", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:be1e76bb-a2bf-4e81-94e0-afda3c6653bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:be1e76bb-a2bf-4e81-94e0-afda3c6653bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3bbfe1ce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3bbfe1ce on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3bbfe1ce on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Thu Oct 11 01:44:44 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4230 (Thu Oct 11 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "13a582df", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Thu Oct 11 01:44:35 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4221 (Thu Oct 11 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cf93ebc4", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f5152f63-9a43-4490-a345-c76acb58b6be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f5152f63-9a43-4490-a345-c76acb58b6be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 42e06cbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 42e06cbc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 42e06cbc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Thu Oct 11 01:44:55 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4241 (Thu Oct 11 01:44:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "65421165", "local_conf_timestamp": 4241, "host-ts": 4240}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Thu Oct 11 01:44:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4232 (Thu Oct 11 01:44:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "37bcf585", "local_conf_timestamp": 4232, "host-ts": 4232}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/391/ Build Number: 391 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.clear_global_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 76, in clear_global_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/392/ Build Number: 392 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/393/ Build Number: 393 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:5e9ccbe1-d202-4ee6-b99d-1599ab86e241:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:5e9ccbe1-d202-4ee6-b99d-1599ab86e241:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running f06d4f8a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f06d4f8a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command f06d4f8a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Oct 12 01:41:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4182 (Fri Oct 12 01:41:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "42fcf518", "local_conf_timestamp": 4182, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Fri Oct 12 01:41:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4190 (Fri Oct 12 01:41:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "281c39e3", "local_conf_timestamp": 4190, "host-ts": 4190}, "global_maintenance": false} lago.ssh: DEBUG: start task:e76573ce-b588-46f7-a28b-69e2a6f6b7bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e76573ce-b588-46f7-a28b-69e2a6f6b7bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f174f9d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command f174f9d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:c1b74fae-ac7d-4516-b68b-50629734d59d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1b74fae-ac7d-4516-b68b-50629734d59d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f27bd58a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f27bd58a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f27bd58a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Fri Oct 12 01:41:25 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4192 (Fri Oct 12 01:41:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0f35f2ae", "local_conf_timestamp": 4192, "host-ts": 4192}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Fri Oct 12 01:41:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4190 (Fri Oct 12 01:41:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "281c39e3", "local_conf_timestamp": 4190, "host-ts": 4190}, "global_maintenance": false} lago.ssh: DEBUG: start task:617b3b35-1e91-45cb-879b-b7e375e5eff3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:617b3b35-1e91-45cb-879b-b7e375e5eff3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fa65bf86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fa65bf86 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fa65bf86 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Fri Oct 12 01:41:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4213 (Fri Oct 12 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2683dac0", "local_conf_timestamp": 4213, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Fri Oct 12 01:41:43 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4210 (Fri Oct 12 01:41:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eda522c3", "local_conf_timestamp": 4210, "host-ts": 4210}, "global_maintenance": false} lago.ssh: DEBUG: start task:1c082449-bd95-42b2-90c1-799122186898:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1c082449-bd95-42b2-90c1-799122186898:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fdb692f0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fdb692f0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fdb692f0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Fri Oct 12 01:41:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4213 (Fri Oct 12 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2683dac0", "local_conf_timestamp": 4213, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Fri Oct 12 01:41:43 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4210 (Fri Oct 12 01:41:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eda522c3", "local_conf_timestamp": 4210, "host-ts": 4210}, "global_maintenance": false} lago.ssh: DEBUG: start task:dd3ebbac-c9cd-4cb0-82c2-74c3c10cbe1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dd3ebbac-c9cd-4cb0-82c2-74c3c10cbe1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 00b73e1e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 00b73e1e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 00b73e1e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Fri Oct 12 01:41:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4222 (Fri Oct 12 01:41:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Fri Oct 12 01:41:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4220 (Fri Oct 12 01:41:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e16e23cc", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false} lago.ssh: DEBUG: start task:4629167d-1cd5-4d12-9fc0-7d0c35d0cece:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4629167d-1cd5-4d12-9fc0-7d0c35d0cece:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 03a7a7a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 03a7a7a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 03a7a7a8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Fri Oct 12 01:41:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4222 (Fri Oct 12 01:41:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Fri Oct 12 01:41:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4220 (Fri Oct 12 01:41:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e16e23cc", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false} lago.ssh: DEBUG: start task:9f1d0228-018b-4692-8641-f95f5ee9cc15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9f1d0228-018b-4692-8641-f95f5ee9cc15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 06ab04d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 06ab04d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 06ab04d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Fri Oct 12 01:42:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4232 (Fri Oct 12 01:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d6939193", "local_conf_timestamp": 4232, "host-ts": 4232}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Fri Oct 12 01:42:03 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4231 (Fri Oct 12 01:42:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "922fec5f", "local_conf_timestamp": 4231, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:2339228c-54fe-4107-8924-475ddc58b1c0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2339228c-54fe-4107-8924-475ddc58b1c0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 09aa3616 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 09aa3616 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 09aa3616 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Fri Oct 12 01:42:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4232 (Fri Oct 12 01:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d6939193", "local_conf_timestamp": 4232, "host-ts": 4232}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Fri Oct 12 01:42:03 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4231 (Fri Oct 12 01:42:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "922fec5f", "local_conf_timestamp": 4231, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:f3e52632-25d3-4825-a3e9-e2b5b04e1cc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f3e52632-25d3-4825-a3e9-e2b5b04e1cc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0c9fad10 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0c9fad10 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0c9fad10 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Fri Oct 12 01:42:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4243 (Fri Oct 12 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5def15c9", "local_conf_timestamp": 4243, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Fri Oct 12 01:42:12 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4240 (Fri Oct 12 01:42:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:3b2e42c3-25ec-4633-9fcc-cb35260c0db2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3b2e42c3-25ec-4633-9fcc-cb35260c0db2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0f882228 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0f882228 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0f882228 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Fri Oct 12 01:42:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4243 (Fri Oct 12 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5def15c9", "local_conf_timestamp": 4243, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Fri Oct 12 01:42:12 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4240 (Fri Oct 12 01:42:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:2e8b9852-51fb-4659-9008-56d1378d11ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2e8b9852-51fb-4659-9008-56d1378d11ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 12cd5ce6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 12cd5ce6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 12cd5ce6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Fri Oct 12 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4253 (Fri Oct 12 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "668f9375", "local_conf_timestamp": 4253, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Fri Oct 12 01:42:23 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4250 (Fri Oct 12 01:42:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:e47b4b71-a0db-47dc-a8d9-0d7de3867b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e47b4b71-a0db-47dc-a8d9-0d7de3867b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 15db6324 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 15db6324 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 15db6324 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Fri Oct 12 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4253 (Fri Oct 12 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "668f9375", "local_conf_timestamp": 4253, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Fri Oct 12 01:42:23 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4250 (Fri Oct 12 01:42:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:6e869956-08c6-4cdb-9885-48e75b886bb2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6e869956-08c6-4cdb-9885-48e75b886bb2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 18be838c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 18be838c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 18be838c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Fri Oct 12 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4264 (Fri Oct 12 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b693f841", "local_conf_timestamp": 4264, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Fri Oct 12 01:42:33 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4260 (Fri Oct 12 01:42:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43510756", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false} lago.ssh: DEBUG: start task:41cee336-e2b1-42a7-9d0c-4a9129d89184:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:41cee336-e2b1-42a7-9d0c-4a9129d89184:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 19d450f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 19d450f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 19d450f8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Fri Oct 12 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4264 (Fri Oct 12 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b693f841", "local_conf_timestamp": 4264, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Fri Oct 12 01:42:33 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4260 (Fri Oct 12 01:42:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43510756", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false} lago.ssh: DEBUG: start task:d6583534-a621-4d40-8bc8-f2fa549b6710:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d6583534-a621-4d40-8bc8-f2fa549b6710:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 20d24bbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 20d24bbc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 20d24bbc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Fri Oct 12 01:42:46 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4273 (Fri Oct 12 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "d511b308", "local_conf_timestamp": 4273, "host-ts": 4273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4271 (Fri Oct 12 01:42:43 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4271 (Fri Oct 12 01:42:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f864efc8", "local_conf_timestamp": 4271, "host-ts": 4271}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:5e9ccbe1-d202-4ee6-b99d-1599ab86e241:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:5e9ccbe1-d202-4ee6-b99d-1599ab86e241:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running f06d4f8a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f06d4f8a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command f06d4f8a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Oct 12 01:41:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4182 (Fri Oct 12 01:41:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "42fcf518", "local_conf_timestamp": 4182, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Fri Oct 12 01:41:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4190 (Fri Oct 12 01:41:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "281c39e3", "local_conf_timestamp": 4190, "host-ts": 4190}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e76573ce-b588-46f7-a28b-69e2a6f6b7bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e76573ce-b588-46f7-a28b-69e2a6f6b7bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f174f9d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command f174f9d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:c1b74fae-ac7d-4516-b68b-50629734d59d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1b74fae-ac7d-4516-b68b-50629734d59d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f27bd58a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f27bd58a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f27bd58a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Fri Oct 12 01:41:25 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4192 (Fri Oct 12 01:41:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0f35f2ae", "local_conf_timestamp": 4192, "host-ts": 4192}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Fri Oct 12 01:41:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4190 (Fri Oct 12 01:41:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "281c39e3", "local_conf_timestamp": 4190, "host-ts": 4190}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:617b3b35-1e91-45cb-879b-b7e375e5eff3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:617b3b35-1e91-45cb-879b-b7e375e5eff3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fa65bf86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fa65bf86 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fa65bf86 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Fri Oct 12 01:41:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4213 (Fri Oct 12 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2683dac0", "local_conf_timestamp": 4213, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Fri Oct 12 01:41:43 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4210 (Fri Oct 12 01:41:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eda522c3", "local_conf_timestamp": 4210, "host-ts": 4210}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1c082449-bd95-42b2-90c1-799122186898:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1c082449-bd95-42b2-90c1-799122186898:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fdb692f0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fdb692f0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fdb692f0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Fri Oct 12 01:41:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4213 (Fri Oct 12 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2683dac0", "local_conf_timestamp": 4213, "host-ts": 4212}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Fri Oct 12 01:41:43 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4210 (Fri Oct 12 01:41:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eda522c3", "local_conf_timestamp": 4210, "host-ts": 4210}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dd3ebbac-c9cd-4cb0-82c2-74c3c10cbe1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dd3ebbac-c9cd-4cb0-82c2-74c3c10cbe1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 00b73e1e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 00b73e1e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 00b73e1e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Fri Oct 12 01:41:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4222 (Fri Oct 12 01:41:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Fri Oct 12 01:41:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4220 (Fri Oct 12 01:41:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e16e23cc", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4629167d-1cd5-4d12-9fc0-7d0c35d0cece:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4629167d-1cd5-4d12-9fc0-7d0c35d0cece:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 03a7a7a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 03a7a7a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 03a7a7a8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Fri Oct 12 01:41:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4222 (Fri Oct 12 01:41:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78a0af51", "local_conf_timestamp": 4222, "host-ts": 4222}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Fri Oct 12 01:41:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4220 (Fri Oct 12 01:41:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e16e23cc", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9f1d0228-018b-4692-8641-f95f5ee9cc15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9f1d0228-018b-4692-8641-f95f5ee9cc15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 06ab04d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 06ab04d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 06ab04d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Fri Oct 12 01:42:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4232 (Fri Oct 12 01:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d6939193", "local_conf_timestamp": 4232, "host-ts": 4232}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Fri Oct 12 01:42:03 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4231 (Fri Oct 12 01:42:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "922fec5f", "local_conf_timestamp": 4231, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2339228c-54fe-4107-8924-475ddc58b1c0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2339228c-54fe-4107-8924-475ddc58b1c0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 09aa3616 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 09aa3616 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 09aa3616 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Fri Oct 12 01:42:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4232 (Fri Oct 12 01:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d6939193", "local_conf_timestamp": 4232, "host-ts": 4232}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Fri Oct 12 01:42:03 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4231 (Fri Oct 12 01:42:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "922fec5f", "local_conf_timestamp": 4231, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f3e52632-25d3-4825-a3e9-e2b5b04e1cc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f3e52632-25d3-4825-a3e9-e2b5b04e1cc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0c9fad10 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0c9fad10 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0c9fad10 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Fri Oct 12 01:42:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4243 (Fri Oct 12 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5def15c9", "local_conf_timestamp": 4243, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Fri Oct 12 01:42:12 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4240 (Fri Oct 12 01:42:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3b2e42c3-25ec-4633-9fcc-cb35260c0db2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3b2e42c3-25ec-4633-9fcc-cb35260c0db2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0f882228 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0f882228 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0f882228 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Fri Oct 12 01:42:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4243 (Fri Oct 12 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5def15c9", "local_conf_timestamp": 4243, "host-ts": 4242}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Fri Oct 12 01:42:12 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4240 (Fri Oct 12 01:42:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4bdcf95c", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2e8b9852-51fb-4659-9008-56d1378d11ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2e8b9852-51fb-4659-9008-56d1378d11ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 12cd5ce6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 12cd5ce6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 12cd5ce6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Fri Oct 12 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4253 (Fri Oct 12 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "668f9375", "local_conf_timestamp": 4253, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Fri Oct 12 01:42:23 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4250 (Fri Oct 12 01:42:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e47b4b71-a0db-47dc-a8d9-0d7de3867b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e47b4b71-a0db-47dc-a8d9-0d7de3867b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 15db6324 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 15db6324 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 15db6324 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Fri Oct 12 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4253 (Fri Oct 12 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "668f9375", "local_conf_timestamp": 4253, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Fri Oct 12 01:42:23 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4250 (Fri Oct 12 01:42:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f9a0659", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6e869956-08c6-4cdb-9885-48e75b886bb2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6e869956-08c6-4cdb-9885-48e75b886bb2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 18be838c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 18be838c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 18be838c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Fri Oct 12 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4264 (Fri Oct 12 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b693f841", "local_conf_timestamp": 4264, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Fri Oct 12 01:42:33 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4260 (Fri Oct 12 01:42:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43510756", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:41cee336-e2b1-42a7-9d0c-4a9129d89184:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:41cee336-e2b1-42a7-9d0c-4a9129d89184:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 19d450f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 19d450f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 19d450f8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Fri Oct 12 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4264 (Fri Oct 12 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b693f841", "local_conf_timestamp": 4264, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Fri Oct 12 01:42:33 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4260 (Fri Oct 12 01:42:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43510756", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d6583534-a621-4d40-8bc8-f2fa549b6710:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d6583534-a621-4d40-8bc8-f2fa549b6710:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 20d24bbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 20d24bbc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 20d24bbc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Fri Oct 12 01:42:46 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4273 (Fri Oct 12 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "d511b308", "local_conf_timestamp": 4273, "host-ts": 4273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4271 (Fri Oct 12 01:42:43 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4271 (Fri Oct 12 01:42:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f864efc8", "local_conf_timestamp": 4271, "host-ts": 4271}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/394/ Build Number: 394 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:d89d53da-5e2d-478d-bd1f-55b8a3db6fa9:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:d89d53da-5e2d-478d-bd1f-55b8a3db6fa9:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running dfa8d10e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dfa8d10e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command dfa8d10e on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4160 (Fri Oct 12 09:40:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4161 (Fri Oct 12 09:40:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecf0f0e6", "local_conf_timestamp": 4161, "host-ts": 4160}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Fri Oct 12 09:40:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4162 (Fri Oct 12 09:40:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7381b9d", "local_conf_timestamp": 4162, "host-ts": 4162}, "global_maintenance": false} lago.ssh: DEBUG: start task:8dbad118-cec0-4805-89b6-a0e787a1e3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8dbad118-cec0-4805-89b6-a0e787a1e3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e0b39cd2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command e0b39cd2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:975b6f17-665e-402d-89aa-c4b35b028bfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:975b6f17-665e-402d-89aa-c4b35b028bfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e1ccc3aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e1ccc3aa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e1ccc3aa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4160 (Fri Oct 12 09:40:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4161 (Fri Oct 12 09:40:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecf0f0e6", "local_conf_timestamp": 4161, "host-ts": 4160}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Fri Oct 12 09:40:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4162 (Fri Oct 12 09:40:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7381b9d", "local_conf_timestamp": 4162, "host-ts": 4162}, "global_maintenance": false} lago.ssh: DEBUG: start task:15028ce3-1968-4d5a-b5ab-78b3be08652d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:15028ce3-1968-4d5a-b5ab-78b3be08652d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e918a4d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e918a4d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e918a4d0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4170 (Fri Oct 12 09:40:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4170 (Fri Oct 12 09:40:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "64ffd763", "local_conf_timestamp": 4170, "host-ts": 4170}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4162 (Fri Oct 12 09:40:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4162 (Fri Oct 12 09:40:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7381b9d", "local_conf_timestamp": 4162, "host-ts": 4162}, "global_maintenance": false} lago.ssh: DEBUG: start task:d9e622d7-8497-4cd0-a28c-e00bf7534de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d9e622d7-8497-4cd0-a28c-e00bf7534de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ec0829d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ec0829d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ec0829d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Fri Oct 12 09:40:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4181 (Fri Oct 12 09:40:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78eba7a2", "local_conf_timestamp": 4181, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Fri Oct 12 09:40:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4172 (Fri Oct 12 09:40:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false} lago.ssh: DEBUG: start task:764c94d2-9c5b-4bd0-8f0b-c807395183b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:764c94d2-9c5b-4bd0-8f0b-c807395183b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ef23eb14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ef23eb14 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ef23eb14 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4180 (Fri Oct 12 09:40:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4181 (Fri Oct 12 09:40:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78eba7a2", "local_conf_timestamp": 4181, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Fri Oct 12 09:40:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4172 (Fri Oct 12 09:40:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false} lago.ssh: DEBUG: start task:de0f4a3b-1c9d-4c50-b070-8e94ca55ab1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:de0f4a3b-1c9d-4c50-b070-8e94ca55ab1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f203e5dc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f203e5dc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f203e5dc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Fri Oct 12 09:41:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4190 (Fri Oct 12 09:41:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Fri Oct 12 09:40:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Fri Oct 12 09:40:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bba4d433", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false} lago.ssh: DEBUG: start task:061f29f5-85e4-48bb-a87b-bb722aded041:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:061f29f5-85e4-48bb-a87b-bb722aded041:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f4e6e998 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f4e6e998 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f4e6e998 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4190 (Fri Oct 12 09:41:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4190 (Fri Oct 12 09:41:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Fri Oct 12 09:40:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Fri Oct 12 09:40:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bba4d433", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false} lago.ssh: DEBUG: start task:53500b86-f5f1-4e09-8a75-2ed47c0c1e25:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:53500b86-f5f1-4e09-8a75-2ed47c0c1e25:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f7fe27ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f7fe27ae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f7fe27ae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4200 (Fri Oct 12 09:41:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4200 (Fri Oct 12 09:41:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Fri Oct 12 09:41:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4192 (Fri Oct 12 09:41:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfe22b36", "local_conf_timestamp": 4192, "host-ts": 4192}, "global_maintenance": false} lago.ssh: DEBUG: start task:f361e823-decf-4a0b-889d-d131b008df27:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f361e823-decf-4a0b-889d-d131b008df27:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fb1d681e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fb1d681e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fb1d681e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4200 (Fri Oct 12 09:41:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4200 (Fri Oct 12 09:41:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Fri Oct 12 09:41:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4192 (Fri Oct 12 09:41:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfe22b36", "local_conf_timestamp": 4192, "host-ts": 4192}, "global_maintenance": false} lago.ssh: DEBUG: start task:2075af06-24ef-471c-b8bd-8ca6252223b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2075af06-24ef-471c-b8bd-8ca6252223b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fdfe6e20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fdfe6e20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fdfe6e20 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Fri Oct 12 09:41:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4210 (Fri Oct 12 09:41:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e5cfdc7", "local_conf_timestamp": 4210, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Fri Oct 12 09:41:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Fri Oct 12 09:41:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false} lago.ssh: DEBUG: start task:749df57e-367d-4d1f-bdec-307ade4291a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:749df57e-367d-4d1f-bdec-307ade4291a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0125636a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0125636a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0125636a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4210 (Fri Oct 12 09:41:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4210 (Fri Oct 12 09:41:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e5cfdc7", "local_conf_timestamp": 4210, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Fri Oct 12 09:41:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Fri Oct 12 09:41:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false} lago.ssh: DEBUG: start task:b0c16946-e9af-434a-8133-905930b35e60:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b0c16946-e9af-434a-8133-905930b35e60:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 04206b64 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 04206b64 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 04206b64 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Fri Oct 12 09:41:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4220 (Fri Oct 12 09:41:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6778b8c0", "local_conf_timestamp": 4220, "host-ts": 4220}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Fri Oct 12 09:41:28 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4212 (Fri Oct 12 09:41:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be42dd00", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false} lago.ssh: DEBUG: start task:d179ea73-9ade-4f46-9380-7b63c3310ce4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d179ea73-9ade-4f46-9380-7b63c3310ce4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 07086944 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 07086944 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 07086944 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Fri Oct 12 09:41:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4220 (Fri Oct 12 09:41:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6778b8c0", "local_conf_timestamp": 4220, "host-ts": 4220}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Fri Oct 12 09:41:28 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4212 (Fri Oct 12 09:41:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be42dd00", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false} lago.ssh: DEBUG: start task:387eadcd-53b0-4d97-a8f1-ada58616b927:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:387eadcd-53b0-4d97-a8f1-ada58616b927:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 09f5e78a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 09f5e78a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 09f5e78a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Fri Oct 12 09:41:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4230 (Fri Oct 12 09:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "26641e43", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Fri Oct 12 09:41:38 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4222 (Fri Oct 12 09:41:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b289dc0f", "local_conf_timestamp": 4222, "host-ts": 4222}, "global_maintenance": false} lago.ssh: DEBUG: start task:17580f89-65bd-4daf-b602-78db4e26f6e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:17580f89-65bd-4daf-b602-78db4e26f6e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0b071f18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0b071f18 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0b071f18 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Fri Oct 12 09:41:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4230 (Fri Oct 12 09:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "26641e43", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Fri Oct 12 09:41:38 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4222 (Fri Oct 12 09:41:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b289dc0f", "local_conf_timestamp": 4222, "host-ts": 4222}, "global_maintenance": false} lago.ssh: DEBUG: start task:98e0e49f-027c-4cf8-9b72-513eeb7591c3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:98e0e49f-027c-4cf8-9b72-513eeb7591c3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 120f1770 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 120f1770 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 120f1770 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Fri Oct 12 09:41:56 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4240 (Fri Oct 12 09:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "b681a127", "local_conf_timestamp": 4240, "host-ts": 4240}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Fri Oct 12 09:41:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4232 (Fri Oct 12 09:41:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1f6353d", "local_conf_timestamp": 4232, "host-ts": 4233}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:d89d53da-5e2d-478d-bd1f-55b8a3db6fa9:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:d89d53da-5e2d-478d-bd1f-55b8a3db6fa9:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running dfa8d10e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dfa8d10e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command dfa8d10e on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4160 (Fri Oct 12 09:40:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4161 (Fri Oct 12 09:40:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecf0f0e6", "local_conf_timestamp": 4161, "host-ts": 4160}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Fri Oct 12 09:40:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4162 (Fri Oct 12 09:40:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7381b9d", "local_conf_timestamp": 4162, "host-ts": 4162}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8dbad118-cec0-4805-89b6-a0e787a1e3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8dbad118-cec0-4805-89b6-a0e787a1e3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e0b39cd2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command e0b39cd2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:975b6f17-665e-402d-89aa-c4b35b028bfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:975b6f17-665e-402d-89aa-c4b35b028bfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e1ccc3aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e1ccc3aa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e1ccc3aa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4160 (Fri Oct 12 09:40:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4161 (Fri Oct 12 09:40:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecf0f0e6", "local_conf_timestamp": 4161, "host-ts": 4160}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Fri Oct 12 09:40:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4162 (Fri Oct 12 09:40:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7381b9d", "local_conf_timestamp": 4162, "host-ts": 4162}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:15028ce3-1968-4d5a-b5ab-78b3be08652d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:15028ce3-1968-4d5a-b5ab-78b3be08652d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e918a4d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e918a4d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e918a4d0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4170 (Fri Oct 12 09:40:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4170 (Fri Oct 12 09:40:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "64ffd763", "local_conf_timestamp": 4170, "host-ts": 4170}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4162 (Fri Oct 12 09:40:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4162 (Fri Oct 12 09:40:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7381b9d", "local_conf_timestamp": 4162, "host-ts": 4162}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d9e622d7-8497-4cd0-a28c-e00bf7534de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d9e622d7-8497-4cd0-a28c-e00bf7534de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ec0829d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ec0829d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ec0829d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Fri Oct 12 09:40:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4181 (Fri Oct 12 09:40:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78eba7a2", "local_conf_timestamp": 4181, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Fri Oct 12 09:40:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4172 (Fri Oct 12 09:40:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:764c94d2-9c5b-4bd0-8f0b-c807395183b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:764c94d2-9c5b-4bd0-8f0b-c807395183b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ef23eb14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ef23eb14 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ef23eb14 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4180 (Fri Oct 12 09:40:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4181 (Fri Oct 12 09:40:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78eba7a2", "local_conf_timestamp": 4181, "host-ts": 4180}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Fri Oct 12 09:40:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4172 (Fri Oct 12 09:40:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:de0f4a3b-1c9d-4c50-b070-8e94ca55ab1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:de0f4a3b-1c9d-4c50-b070-8e94ca55ab1e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f203e5dc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f203e5dc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f203e5dc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Fri Oct 12 09:41:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4190 (Fri Oct 12 09:41:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Fri Oct 12 09:40:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Fri Oct 12 09:40:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bba4d433", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:061f29f5-85e4-48bb-a87b-bb722aded041:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:061f29f5-85e4-48bb-a87b-bb722aded041:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f4e6e998 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f4e6e998 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f4e6e998 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4190 (Fri Oct 12 09:41:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4190 (Fri Oct 12 09:41:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a1dfa9f6", "local_conf_timestamp": 4190, "host-ts": 4190}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Fri Oct 12 09:40:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Fri Oct 12 09:40:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bba4d433", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:53500b86-f5f1-4e09-8a75-2ed47c0c1e25:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:53500b86-f5f1-4e09-8a75-2ed47c0c1e25:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f7fe27ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f7fe27ae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f7fe27ae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4200 (Fri Oct 12 09:41:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4200 (Fri Oct 12 09:41:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Fri Oct 12 09:41:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4192 (Fri Oct 12 09:41:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfe22b36", "local_conf_timestamp": 4192, "host-ts": 4192}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f361e823-decf-4a0b-889d-d131b008df27:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f361e823-decf-4a0b-889d-d131b008df27:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fb1d681e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fb1d681e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fb1d681e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4200 (Fri Oct 12 09:41:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4200 (Fri Oct 12 09:41:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e06fc305", "local_conf_timestamp": 4200, "host-ts": 4200}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Fri Oct 12 09:41:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4192 (Fri Oct 12 09:41:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfe22b36", "local_conf_timestamp": 4192, "host-ts": 4192}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2075af06-24ef-471c-b8bd-8ca6252223b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2075af06-24ef-471c-b8bd-8ca6252223b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fdfe6e20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fdfe6e20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fdfe6e20 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Fri Oct 12 09:41:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4210 (Fri Oct 12 09:41:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e5cfdc7", "local_conf_timestamp": 4210, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Fri Oct 12 09:41:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Fri Oct 12 09:41:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:749df57e-367d-4d1f-bdec-307ade4291a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:749df57e-367d-4d1f-bdec-307ade4291a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0125636a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0125636a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0125636a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4210 (Fri Oct 12 09:41:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4210 (Fri Oct 12 09:41:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e5cfdc7", "local_conf_timestamp": 4210, "host-ts": 4210}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Fri Oct 12 09:41:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Fri Oct 12 09:41:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b0c16946-e9af-434a-8133-905930b35e60:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b0c16946-e9af-434a-8133-905930b35e60:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 04206b64 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 04206b64 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 04206b64 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Fri Oct 12 09:41:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4220 (Fri Oct 12 09:41:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6778b8c0", "local_conf_timestamp": 4220, "host-ts": 4220}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Fri Oct 12 09:41:28 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4212 (Fri Oct 12 09:41:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be42dd00", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d179ea73-9ade-4f46-9380-7b63c3310ce4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d179ea73-9ade-4f46-9380-7b63c3310ce4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 07086944 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 07086944 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 07086944 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Fri Oct 12 09:41:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4220 (Fri Oct 12 09:41:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6778b8c0", "local_conf_timestamp": 4220, "host-ts": 4220}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Fri Oct 12 09:41:28 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4212 (Fri Oct 12 09:41:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be42dd00", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:387eadcd-53b0-4d97-a8f1-ada58616b927:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:387eadcd-53b0-4d97-a8f1-ada58616b927:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 09f5e78a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 09f5e78a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 09f5e78a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Fri Oct 12 09:41:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4230 (Fri Oct 12 09:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "26641e43", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Fri Oct 12 09:41:38 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4222 (Fri Oct 12 09:41:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b289dc0f", "local_conf_timestamp": 4222, "host-ts": 4222}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:17580f89-65bd-4daf-b602-78db4e26f6e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:17580f89-65bd-4daf-b602-78db4e26f6e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0b071f18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0b071f18 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0b071f18 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Fri Oct 12 09:41:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4230 (Fri Oct 12 09:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "26641e43", "local_conf_timestamp": 4230, "host-ts": 4230}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Fri Oct 12 09:41:38 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4222 (Fri Oct 12 09:41:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b289dc0f", "local_conf_timestamp": 4222, "host-ts": 4222}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:98e0e49f-027c-4cf8-9b72-513eeb7591c3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:98e0e49f-027c-4cf8-9b72-513eeb7591c3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 120f1770 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 120f1770 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 120f1770 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Fri Oct 12 09:41:56 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4240 (Fri Oct 12 09:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "b681a127", "local_conf_timestamp": 4240, "host-ts": 4240}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Fri Oct 12 09:41:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4232 (Fri Oct 12 09:41:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1f6353d", "local_conf_timestamp": 4232, "host-ts": 4233}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/395/ Build Number: 395 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #395 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:da3a0828-358e-4d7d-855c-b1f6aafee925:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:da3a0828-358e-4d7d-855c-b1f6aafee925:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 79192030 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 79192030 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 79192030 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dbf5a4a2", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:8012f639-3674-422e-9f52-803a1af1bba5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8012f639-3674-422e-9f52-803a1af1bba5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7a33c3b2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 7a33c3b2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:aef26a6c-67ed-48ea-9cb2-d36584df9eb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aef26a6c-67ed-48ea-9cb2-d36584df9eb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7b481d0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7b481d0c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7b481d0c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dbf5a4a2", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:faf742b7-ffe2-4e5c-9046-b3204379c411:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:faf742b7-ffe2-4e5c-9046-b3204379c411:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8268ad04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8268ad04 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8268ad04 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Fri Oct 12 17:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4191 (Fri Oct 12 17:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24fdb3b1", "local_conf_timestamp": 4191, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Fri Oct 12 17:44:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Fri Oct 12 17:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:4b7bcbdf-ee05-4a94-9cff-e70bd55af810:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4b7bcbdf-ee05-4a94-9cff-e70bd55af810:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 85bc7738 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 85bc7738 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 85bc7738 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Fri Oct 12 17:44:50 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4201 (Fri Oct 12 17:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "023b4bed", "local_conf_timestamp": 4201, "host-ts": 4201}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Fri Oct 12 17:44:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Fri Oct 12 17:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed339cfa", "local_conf_timestamp": 4202, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:898cf576-48b7-418c-b67c-9498ce88bf12:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:898cf576-48b7-418c-b67c-9498ce88bf12:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 88c63f04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 88c63f04 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 88c63f04 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Fri Oct 12 17:44:50 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4201 (Fri Oct 12 17:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "023b4bed", "local_conf_timestamp": 4201, "host-ts": 4201}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Fri Oct 12 17:44:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Fri Oct 12 17:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed339cfa", "local_conf_timestamp": 4202, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:c7393e6e-ae72-41a1-82ab-e8ec3ec9ee74:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c7393e6e-ae72-41a1-82ab-e8ec3ec9ee74:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8bfbd300 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8bfbd300 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8bfbd300 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac08752f", "local_conf_timestamp": 4211, "host-ts": 4211}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:912091e5-9919-44eb-ba25-54540d697ac3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:912091e5-9919-44eb-ba25-54540d697ac3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8f565ce6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8f565ce6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8f565ce6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac08752f", "local_conf_timestamp": 4211, "host-ts": 4211}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:1afd959c-2708-43c5-b75e-a0a732313e92:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1afd959c-2708-43c5-b75e-a0a732313e92:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9241a366 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9241a366 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9241a366 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:264bf386-4e96-4a7f-a961-0e1926f1acfc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:264bf386-4e96-4a7f-a961-0e1926f1acfc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9535892a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9535892a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9535892a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:503d91f8-56cf-46ca-8c95-9c4df249c0b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:503d91f8-56cf-46ca-8c95-9c4df249c0b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9837a338 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9837a338 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9837a338 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cbd514c1", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false} lago.ssh: DEBUG: start task:ca0d44c8-0ee5-4522-a1b8-988ac1992d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ca0d44c8-0ee5-4522-a1b8-988ac1992d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9b718000 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9b718000 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9b718000 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cbd514c1", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false} lago.ssh: DEBUG: start task:8c70d22a-72af-4916-a083-1502019ea5c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8c70d22a-72af-4916-a083-1502019ea5c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9e47611e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9e47611e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9e47611e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Oct 12 17:45:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4242 (Fri Oct 12 17:45:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e6deb9c", "local_conf_timestamp": 4242, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Oct 12 17:45:29 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4241 (Fri Oct 12 17:45:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false} lago.ssh: DEBUG: start task:eca9f990-3e4e-4b6c-92dc-abade532e7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:eca9f990-3e4e-4b6c-92dc-abade532e7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a139bea8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a139bea8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a139bea8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Oct 12 17:45:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4242 (Fri Oct 12 17:45:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e6deb9c", "local_conf_timestamp": 4242, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Oct 12 17:45:29 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4241 (Fri Oct 12 17:45:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false} lago.ssh: DEBUG: start task:8b3fa397-f6b1-46b2-8c7f-d4469fa88e43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8b3fa397-f6b1-46b2-8c7f-d4469fa88e43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a426a748 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a426a748 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a426a748 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5dabca8c", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9bdb69f7", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false} lago.ssh: DEBUG: start task:ec021552-62bc-4768-a055-62f93cfc10c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ec021552-62bc-4768-a055-62f93cfc10c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a5483664 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a5483664 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a5483664 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5dabca8c", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9bdb69f7", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:da3a0828-358e-4d7d-855c-b1f6aafee925:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:da3a0828-358e-4d7d-855c-b1f6aafee925:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 79192030 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 79192030 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 79192030 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dbf5a4a2", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8012f639-3674-422e-9f52-803a1af1bba5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8012f639-3674-422e-9f52-803a1af1bba5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7a33c3b2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 7a33c3b2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:aef26a6c-67ed-48ea-9cb2-d36584df9eb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aef26a6c-67ed-48ea-9cb2-d36584df9eb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7b481d0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7b481d0c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7b481d0c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dbf5a4a2", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Oct 12 17:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4181 (Fri Oct 12 17:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:faf742b7-ffe2-4e5c-9046-b3204379c411:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:faf742b7-ffe2-4e5c-9046-b3204379c411:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8268ad04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8268ad04 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8268ad04 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Fri Oct 12 17:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4191 (Fri Oct 12 17:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24fdb3b1", "local_conf_timestamp": 4191, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Fri Oct 12 17:44:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Fri Oct 12 17:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4b7bcbdf-ee05-4a94-9cff-e70bd55af810:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4b7bcbdf-ee05-4a94-9cff-e70bd55af810:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 85bc7738 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 85bc7738 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 85bc7738 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Fri Oct 12 17:44:50 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4201 (Fri Oct 12 17:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "023b4bed", "local_conf_timestamp": 4201, "host-ts": 4201}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Fri Oct 12 17:44:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Fri Oct 12 17:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed339cfa", "local_conf_timestamp": 4202, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:898cf576-48b7-418c-b67c-9498ce88bf12:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:898cf576-48b7-418c-b67c-9498ce88bf12:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 88c63f04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 88c63f04 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 88c63f04 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Fri Oct 12 17:44:50 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4201 (Fri Oct 12 17:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "023b4bed", "local_conf_timestamp": 4201, "host-ts": 4201}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Fri Oct 12 17:44:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Fri Oct 12 17:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ed339cfa", "local_conf_timestamp": 4202, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c7393e6e-ae72-41a1-82ab-e8ec3ec9ee74:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c7393e6e-ae72-41a1-82ab-e8ec3ec9ee74:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8bfbd300 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8bfbd300 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8bfbd300 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac08752f", "local_conf_timestamp": 4211, "host-ts": 4211}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:912091e5-9919-44eb-ba25-54540d697ac3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:912091e5-9919-44eb-ba25-54540d697ac3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8f565ce6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8f565ce6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8f565ce6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac08752f", "local_conf_timestamp": 4211, "host-ts": 4211}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Oct 12 17:44:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Fri Oct 12 17:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1afd959c-2708-43c5-b75e-a0a732313e92:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1afd959c-2708-43c5-b75e-a0a732313e92:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9241a366 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9241a366 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9241a366 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:264bf386-4e96-4a7f-a961-0e1926f1acfc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:264bf386-4e96-4a7f-a961-0e1926f1acfc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9535892a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9535892a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9535892a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "852c3028", "local_conf_timestamp": 4221, "host-ts": 4221}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Oct 12 17:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4221 (Fri Oct 12 17:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:503d91f8-56cf-46ca-8c95-9c4df249c0b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:503d91f8-56cf-46ca-8c95-9c4df249c0b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9837a338 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9837a338 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9837a338 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cbd514c1", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ca0d44c8-0ee5-4522-a1b8-988ac1992d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ca0d44c8-0ee5-4522-a1b8-988ac1992d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9b718000 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9b718000 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9b718000 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b1f0eea", "local_conf_timestamp": 4231, "host-ts": 4231}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Oct 12 17:45:19 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4231 (Fri Oct 12 17:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cbd514c1", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8c70d22a-72af-4916-a083-1502019ea5c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8c70d22a-72af-4916-a083-1502019ea5c9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9e47611e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9e47611e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9e47611e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Oct 12 17:45:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4242 (Fri Oct 12 17:45:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e6deb9c", "local_conf_timestamp": 4242, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Oct 12 17:45:29 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4241 (Fri Oct 12 17:45:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:eca9f990-3e4e-4b6c-92dc-abade532e7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:eca9f990-3e4e-4b6c-92dc-abade532e7b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a139bea8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a139bea8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a139bea8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Oct 12 17:45:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4242 (Fri Oct 12 17:45:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4e6deb9c", "local_conf_timestamp": 4242, "host-ts": 4241}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Oct 12 17:45:29 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4241 (Fri Oct 12 17:45:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8b3fa397-f6b1-46b2-8c7f-d4469fa88e43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8b3fa397-f6b1-46b2-8c7f-d4469fa88e43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a426a748 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a426a748 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a426a748 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5dabca8c", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9bdb69f7", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ec021552-62bc-4768-a055-62f93cfc10c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ec021552-62bc-4768-a055-62f93cfc10c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a5483664 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a5483664 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a5483664 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5dabca8c", "local_conf_timestamp": 4251, "host-ts": 4251}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Oct 12 17:45:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4251 (Fri Oct 12 17:45:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9bdb69f7", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/396/ Build Number: 396 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #395 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #396 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:92984b83-8ac4-4390-b8db-fab332efd303:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:92984b83-8ac4-4390-b8db-fab332efd303:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 3c6eb576 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3c6eb576 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 3c6eb576 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4167 (Sat Oct 13 01:42:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4167 (Sat Oct 13 01:42:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3bf41c38", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Sat Oct 13 01:42:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4172 (Sat Oct 13 01:42:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false} lago.ssh: DEBUG: start task:0425e165-cbe8-4092-9f52-e47497928de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0425e165-cbe8-4092-9f52-e47497928de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3d74e864 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 3d74e864 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:e496d289-deb2-4736-b4bb-9204269f9978:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e496d289-deb2-4736-b4bb-9204269f9978:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3e82f44e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3e82f44e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3e82f44e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Sat Oct 13 01:42:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4177 (Sat Oct 13 01:42:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7935c07", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Sat Oct 13 01:42:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4172 (Sat Oct 13 01:42:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false} lago.ssh: DEBUG: start task:955a1007-9cd8-433a-b93c-0c86ecca896e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:955a1007-9cd8-433a-b93c-0c86ecca896e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 45dbce82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 45dbce82 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 45dbce82 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4187 (Sat Oct 13 01:42:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4187 (Sat Oct 13 01:42:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9d4dfe8", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Sat Oct 13 01:42:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Sat Oct 13 01:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cc99e1ab", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false} lago.ssh: DEBUG: start task:81f09ed3-396c-4de9-a6d7-35e3cf71267e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:81f09ed3-396c-4de9-a6d7-35e3cf71267e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 49a10fdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 49a10fdc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 49a10fdc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4197 (Sat Oct 13 01:42:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4197 (Sat Oct 13 01:42:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7f019970", "local_conf_timestamp": 4197, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Sat Oct 13 01:42:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4193 (Sat Oct 13 01:42:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8e51ba0", "local_conf_timestamp": 4193, "host-ts": 4192}, "global_maintenance": false} lago.ssh: DEBUG: start task:37a1a751-8932-41c2-af76-479733384384:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:37a1a751-8932-41c2-af76-479733384384:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4ad632f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4ad632f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4ad632f6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4197 (Sat Oct 13 01:42:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4197 (Sat Oct 13 01:42:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7f019970", "local_conf_timestamp": 4197, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Sat Oct 13 01:42:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4193 (Sat Oct 13 01:42:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8e51ba0", "local_conf_timestamp": 4193, "host-ts": 4192}, "global_maintenance": false} lago.ssh: DEBUG: start task:7a793ff4-f806-4a5b-8587-e728e5ba7e0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7a793ff4-f806-4a5b-8587-e728e5ba7e0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 522d71a4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 522d71a4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 522d71a4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Sat Oct 13 01:42:54 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Sat Oct 13 01:42:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Sat Oct 13 01:42:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Sat Oct 13 01:42:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false} lago.ssh: DEBUG: start task:4bba4ad3-25e2-43c0-9e65-efe37c75b0f5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4bba4ad3-25e2-43c0-9e65-efe37c75b0f5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5568556e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5568556e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5568556e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Sat Oct 13 01:43:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4217 (Sat Oct 13 01:43:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c604d9c", "local_conf_timestamp": 4217, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Sat Oct 13 01:42:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4212 (Sat Oct 13 01:42:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false} lago.ssh: DEBUG: start task:dbcc3120-2610-49b0-b97e-71c0bf4072af:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dbcc3120-2610-49b0-b97e-71c0bf4072af:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 585f9e80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 585f9e80 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 585f9e80 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Sat Oct 13 01:43:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4217 (Sat Oct 13 01:43:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c604d9c", "local_conf_timestamp": 4217, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Sat Oct 13 01:42:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4212 (Sat Oct 13 01:42:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false} lago.ssh: DEBUG: start task:8d00eb73-8cdf-4ba1-92e0-b82f4efd3ba7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8d00eb73-8cdf-4ba1-92e0-b82f4efd3ba7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5ba10c5a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5ba10c5a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5ba10c5a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Sat Oct 13 01:43:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4227 (Sat Oct 13 01:43:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a544089b", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Sat Oct 13 01:43:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4222 (Sat Oct 13 01:43:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "01ad0481", "local_conf_timestamp": 4222, "host-ts": 4222}, "global_maintenance": false} lago.ssh: DEBUG: start task:6433cd3c-c9bf-461b-a49a-8f042f373d06:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6433cd3c-c9bf-461b-a49a-8f042f373d06:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5ec9914a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5ec9914a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5ec9914a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Sat Oct 13 01:43:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4227 (Sat Oct 13 01:43:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a544089b", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Sat Oct 13 01:43:19 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4233 (Sat Oct 13 01:43:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1c8139c", "local_conf_timestamp": 4233, "host-ts": 4232}, "global_maintenance": false} lago.ssh: DEBUG: start task:f80027ff-b30a-4635-84fc-70f911f385c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f80027ff-b30a-4635-84fc-70f911f385c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 61ec5b32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 61ec5b32 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 61ec5b32 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Sat Oct 13 01:43:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Sat Oct 13 01:43:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0b773659", "local_conf_timestamp": 4237, "host-ts": 4237}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Sat Oct 13 01:43:19 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4233 (Sat Oct 13 01:43:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1c8139c", "local_conf_timestamp": 4233, "host-ts": 4232}, "global_maintenance": false} lago.ssh: DEBUG: start task:4daa5725-25ce-4ed2-9d87-f827fc71cc22:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4daa5725-25ce-4ed2-9d87-f827fc71cc22:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6539ab32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6539ab32 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6539ab32 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Sat Oct 13 01:43:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Sat Oct 13 01:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Sat Oct 13 01:43:28 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4242 (Sat Oct 13 01:43:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ab1fde11", "local_conf_timestamp": 4242, "host-ts": 4242}, "global_maintenance": false} lago.ssh: DEBUG: start task:ee21f1f9-90a2-4de6-bea2-07ba6fac7aa3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ee21f1f9-90a2-4de6-bea2-07ba6fac7aa3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 684e7316 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 684e7316 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 684e7316 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Sat Oct 13 01:43:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Sat Oct 13 01:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Sat Oct 13 01:43:28 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4242 (Sat Oct 13 01:43:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ab1fde11", "local_conf_timestamp": 4242, "host-ts": 4242}, "global_maintenance": false} lago.ssh: DEBUG: start task:18e6eec4-1c48-4760-a29d-5ac44874f243:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:18e6eec4-1c48-4760-a29d-5ac44874f243:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6b27c9c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6b27c9c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6b27c9c0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Sat Oct 13 01:43:44 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4257 (Sat Oct 13 01:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ddeffd1", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4252 (Sat Oct 13 01:43:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4252 (Sat Oct 13 01:43:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "937c7fc4", "local_conf_timestamp": 4252, "host-ts": 4252}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:92984b83-8ac4-4390-b8db-fab332efd303:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:92984b83-8ac4-4390-b8db-fab332efd303:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 3c6eb576 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3c6eb576 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 3c6eb576 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4167 (Sat Oct 13 01:42:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4167 (Sat Oct 13 01:42:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3bf41c38", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Sat Oct 13 01:42:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4172 (Sat Oct 13 01:42:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0425e165-cbe8-4092-9f52-e47497928de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0425e165-cbe8-4092-9f52-e47497928de6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3d74e864 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 3d74e864 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:e496d289-deb2-4736-b4bb-9204269f9978:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e496d289-deb2-4736-b4bb-9204269f9978:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3e82f44e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3e82f44e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3e82f44e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Sat Oct 13 01:42:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4177 (Sat Oct 13 01:42:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7935c07", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Sat Oct 13 01:42:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4172 (Sat Oct 13 01:42:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:955a1007-9cd8-433a-b93c-0c86ecca896e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:955a1007-9cd8-433a-b93c-0c86ecca896e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 45dbce82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 45dbce82 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 45dbce82 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4187 (Sat Oct 13 01:42:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4187 (Sat Oct 13 01:42:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9d4dfe8", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Sat Oct 13 01:42:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Sat Oct 13 01:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cc99e1ab", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:81f09ed3-396c-4de9-a6d7-35e3cf71267e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:81f09ed3-396c-4de9-a6d7-35e3cf71267e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 49a10fdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 49a10fdc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 49a10fdc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4197 (Sat Oct 13 01:42:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4197 (Sat Oct 13 01:42:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7f019970", "local_conf_timestamp": 4197, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Sat Oct 13 01:42:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4193 (Sat Oct 13 01:42:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8e51ba0", "local_conf_timestamp": 4193, "host-ts": 4192}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:37a1a751-8932-41c2-af76-479733384384:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:37a1a751-8932-41c2-af76-479733384384:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4ad632f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4ad632f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4ad632f6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4197 (Sat Oct 13 01:42:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4197 (Sat Oct 13 01:42:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7f019970", "local_conf_timestamp": 4197, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Sat Oct 13 01:42:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4193 (Sat Oct 13 01:42:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8e51ba0", "local_conf_timestamp": 4193, "host-ts": 4192}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7a793ff4-f806-4a5b-8587-e728e5ba7e0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7a793ff4-f806-4a5b-8587-e728e5ba7e0e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 522d71a4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 522d71a4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 522d71a4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Sat Oct 13 01:42:54 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Sat Oct 13 01:42:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Sat Oct 13 01:42:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Sat Oct 13 01:42:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4bba4ad3-25e2-43c0-9e65-efe37c75b0f5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4bba4ad3-25e2-43c0-9e65-efe37c75b0f5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5568556e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5568556e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5568556e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Sat Oct 13 01:43:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4217 (Sat Oct 13 01:43:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c604d9c", "local_conf_timestamp": 4217, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Sat Oct 13 01:42:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4212 (Sat Oct 13 01:42:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dbcc3120-2610-49b0-b97e-71c0bf4072af:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dbcc3120-2610-49b0-b97e-71c0bf4072af:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 585f9e80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 585f9e80 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 585f9e80 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Sat Oct 13 01:43:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4217 (Sat Oct 13 01:43:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c604d9c", "local_conf_timestamp": 4217, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Sat Oct 13 01:42:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4212 (Sat Oct 13 01:42:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8d00eb73-8cdf-4ba1-92e0-b82f4efd3ba7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8d00eb73-8cdf-4ba1-92e0-b82f4efd3ba7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5ba10c5a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5ba10c5a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5ba10c5a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Sat Oct 13 01:43:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4227 (Sat Oct 13 01:43:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a544089b", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Sat Oct 13 01:43:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4222 (Sat Oct 13 01:43:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "01ad0481", "local_conf_timestamp": 4222, "host-ts": 4222}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6433cd3c-c9bf-461b-a49a-8f042f373d06:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6433cd3c-c9bf-461b-a49a-8f042f373d06:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5ec9914a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5ec9914a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5ec9914a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Sat Oct 13 01:43:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4227 (Sat Oct 13 01:43:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a544089b", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Sat Oct 13 01:43:19 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4233 (Sat Oct 13 01:43:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1c8139c", "local_conf_timestamp": 4233, "host-ts": 4232}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f80027ff-b30a-4635-84fc-70f911f385c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f80027ff-b30a-4635-84fc-70f911f385c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 61ec5b32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 61ec5b32 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 61ec5b32 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Sat Oct 13 01:43:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Sat Oct 13 01:43:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0b773659", "local_conf_timestamp": 4237, "host-ts": 4237}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Sat Oct 13 01:43:19 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4233 (Sat Oct 13 01:43:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1c8139c", "local_conf_timestamp": 4233, "host-ts": 4232}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4daa5725-25ce-4ed2-9d87-f827fc71cc22:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4daa5725-25ce-4ed2-9d87-f827fc71cc22:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6539ab32 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6539ab32 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6539ab32 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Sat Oct 13 01:43:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Sat Oct 13 01:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Sat Oct 13 01:43:28 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4242 (Sat Oct 13 01:43:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ab1fde11", "local_conf_timestamp": 4242, "host-ts": 4242}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ee21f1f9-90a2-4de6-bea2-07ba6fac7aa3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ee21f1f9-90a2-4de6-bea2-07ba6fac7aa3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 684e7316 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 684e7316 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 684e7316 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Sat Oct 13 01:43:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Sat Oct 13 01:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Sat Oct 13 01:43:28 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4242 (Sat Oct 13 01:43:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ab1fde11", "local_conf_timestamp": 4242, "host-ts": 4242}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:18e6eec4-1c48-4760-a29d-5ac44874f243:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:18e6eec4-1c48-4760-a29d-5ac44874f243:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6b27c9c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6b27c9c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6b27c9c0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Sat Oct 13 01:43:44 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4257 (Sat Oct 13 01:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ddeffd1", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4252 (Sat Oct 13 01:43:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4252 (Sat Oct 13 01:43:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "937c7fc4", "local_conf_timestamp": 4252, "host-ts": 4252}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/397/ Build Number: 397 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #395 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #396 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #397 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:2e60f807-1de0-44b3-a403-c38a2bf75bf2:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:2e60f807-1de0-44b3-a403-c38a2bf75bf2:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 32f59666 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 32f59666 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 32f59666 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3953 (Sat Oct 13 09:41:30 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3953 (Sat Oct 13 09:41:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5300571d", "local_conf_timestamp": 3953, "host-ts": 3953}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3960 (Sat Oct 13 09:41:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3960 (Sat Oct 13 09:41:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4856fb39", "local_conf_timestamp": 3960, "host-ts": 3960}, "global_maintenance": false} lago.ssh: DEBUG: start task:2eef2c49-c439-4c49-807a-fd834697673e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2eef2c49-c439-4c49-807a-fd834697673e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 34069ac8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 34069ac8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:470d81cc-1437-4e1c-bf28-bfa1eee53b5b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:470d81cc-1437-4e1c-bf28-bfa1eee53b5b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3517905c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3517905c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3517905c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3963 (Sat Oct 13 09:41:40 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3963 (Sat Oct 13 09:41:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecd8911d", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3970 (Sat Oct 13 09:41:47 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3970 (Sat Oct 13 09:41:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4c10043c", "local_conf_timestamp": 3970, "host-ts": 3970}, "global_maintenance": false} lago.ssh: DEBUG: start task:55f1931e-70e2-4559-adfa-0e051770ff4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:55f1931e-70e2-4559-adfa-0e051770ff4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3c8217ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3c8217ae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3c8217ae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3984 (Sat Oct 13 09:42:01 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3984 (Sat Oct 13 09:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "83d92228", "local_conf_timestamp": 3984, "host-ts": 3984}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3980 (Sat Oct 13 09:41:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3980 (Sat Oct 13 09:41:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73f7010f", "local_conf_timestamp": 3980, "host-ts": 3980}, "global_maintenance": false} lago.ssh: DEBUG: start task:509e07e6-046d-4bac-8891-edbe344d97ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:509e07e6-046d-4bac-8891-edbe344d97ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 40557470 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 40557470 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 40557470 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3984 (Sat Oct 13 09:42:01 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3984 (Sat Oct 13 09:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "83d92228", "local_conf_timestamp": 3984, "host-ts": 3984}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3980 (Sat Oct 13 09:41:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3980 (Sat Oct 13 09:41:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73f7010f", "local_conf_timestamp": 3980, "host-ts": 3980}, "global_maintenance": false} lago.ssh: DEBUG: start task:aa6fc850-c84e-46cf-bc99-6a9be33616b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aa6fc850-c84e-46cf-bc99-6a9be33616b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 43581268 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 43581268 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 43581268 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3994 (Sat Oct 13 09:42:11 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Sat Oct 13 09:42:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3990 (Sat Oct 13 09:42:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3990 (Sat Oct 13 09:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "008ccb92", "local_conf_timestamp": 3990, "host-ts": 3990}, "global_maintenance": false} lago.ssh: DEBUG: start task:7fcd8428-6744-4b55-b56f-f1dc619b66f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7fcd8428-6744-4b55-b56f-f1dc619b66f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4637f4f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4637f4f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4637f4f8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3994 (Sat Oct 13 09:42:11 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Sat Oct 13 09:42:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3990 (Sat Oct 13 09:42:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3990 (Sat Oct 13 09:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "008ccb92", "local_conf_timestamp": 3990, "host-ts": 3990}, "global_maintenance": false} lago.ssh: DEBUG: start task:45cd725b-a4ba-47f5-be0f-38abdf20e35b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:45cd725b-a4ba-47f5-be0f-38abdf20e35b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4951e252 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4951e252 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4951e252 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4004 (Sat Oct 13 09:42:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4004 (Sat Oct 13 09:42:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3456cd63", "local_conf_timestamp": 4004, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4000 (Sat Oct 13 09:42:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4000 (Sat Oct 13 09:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "093eb926", "local_conf_timestamp": 4000, "host-ts": 4000}, "global_maintenance": false} lago.ssh: DEBUG: start task:d4a9e5e7-3bdf-4f52-9f4a-491cad5dee66:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d4a9e5e7-3bdf-4f52-9f4a-491cad5dee66:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4c35b660 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4c35b660 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4c35b660 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4004 (Sat Oct 13 09:42:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4004 (Sat Oct 13 09:42:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3456cd63", "local_conf_timestamp": 4004, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4000 (Sat Oct 13 09:42:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4000 (Sat Oct 13 09:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "093eb926", "local_conf_timestamp": 4000, "host-ts": 4000}, "global_maintenance": false} lago.ssh: DEBUG: start task:132ae362-3d39-4c9a-abd7-55f4e8a95349:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:132ae362-3d39-4c9a-abd7-55f4e8a95349:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4f123fa2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4f123fa2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4f123fa2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4014 (Sat Oct 13 09:42:31 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4014 (Sat Oct 13 09:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9a65f3a1", "local_conf_timestamp": 4014, "host-ts": 4014}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4010 (Sat Oct 13 09:42:27 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4010 (Sat Oct 13 09:42:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be5c9ead", "local_conf_timestamp": 4010, "host-ts": 4010}, "global_maintenance": false} lago.ssh: DEBUG: start task:e5c3ff75-7c6d-4b8f-8467-06c1a7e17ecd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e5c3ff75-7c6d-4b8f-8467-06c1a7e17ecd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 523a6d8a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 523a6d8a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 523a6d8a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4014 (Sat Oct 13 09:42:31 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4014 (Sat Oct 13 09:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9a65f3a1", "local_conf_timestamp": 4014, "host-ts": 4014}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4010 (Sat Oct 13 09:42:27 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4010 (Sat Oct 13 09:42:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be5c9ead", "local_conf_timestamp": 4010, "host-ts": 4010}, "global_maintenance": false} lago.ssh: DEBUG: start task:c48c94d4-c995-4b70-9bc0-192cf9ecbc50:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c48c94d4-c995-4b70-9bc0-192cf9ecbc50:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 553c4daa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 553c4daa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 553c4daa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4024 (Sat Oct 13 09:42:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4024 (Sat Oct 13 09:42:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b341b6a6", "local_conf_timestamp": 4024, "host-ts": 4024}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4020 (Sat Oct 13 09:42:37 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4020 (Sat Oct 13 09:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b2979fa2", "local_conf_timestamp": 4020, "host-ts": 4020}, "global_maintenance": false} lago.ssh: DEBUG: start task:f81612e6-7892-40b4-b170-de14d969e414:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f81612e6-7892-40b4-b170-de14d969e414:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 584aaff0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 584aaff0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 584aaff0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4024 (Sat Oct 13 09:42:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4024 (Sat Oct 13 09:42:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b341b6a6", "local_conf_timestamp": 4024, "host-ts": 4024}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4020 (Sat Oct 13 09:42:37 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4020 (Sat Oct 13 09:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b2979fa2", "local_conf_timestamp": 4020, "host-ts": 4020}, "global_maintenance": false} lago.ssh: DEBUG: start task:fa20f926-8b1c-4299-906b-ebd851133cb9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fa20f926-8b1c-4299-906b-ebd851133cb9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5b49bb56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5b49bb56 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5b49bb56 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4034 (Sat Oct 13 09:42:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4034 (Sat Oct 13 09:42:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "35aa34ac", "local_conf_timestamp": 4034, "host-ts": 4034}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4030 (Sat Oct 13 09:42:47 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4031 (Sat Oct 13 09:42:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1d65031", "local_conf_timestamp": 4031, "host-ts": 4030}, "global_maintenance": false} lago.ssh: DEBUG: start task:702bdfca-69b1-4e2a-ab38-879b0e674c97:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:702bdfca-69b1-4e2a-ab38-879b0e674c97:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5c53f7aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5c53f7aa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5c53f7aa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4034 (Sat Oct 13 09:42:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4034 (Sat Oct 13 09:42:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "35aa34ac", "local_conf_timestamp": 4034, "host-ts": 4034}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4030 (Sat Oct 13 09:42:47 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4031 (Sat Oct 13 09:42:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1d65031", "local_conf_timestamp": 4031, "host-ts": 4030}, "global_maintenance": false} lago.ssh: DEBUG: start task:2290d4e9-df48-4897-b1cf-b5046621634f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2290d4e9-df48-4897-b1cf-b5046621634f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 634fb012 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 634fb012 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 634fb012 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4044 (Sat Oct 13 09:43:01 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4044 (Sat Oct 13 09:43:01 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "2d6a3e8b", "local_conf_timestamp": 4044, "host-ts": 4044}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4041 (Sat Oct 13 09:42:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4041 (Sat Oct 13 09:42:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1b9bb73", "local_conf_timestamp": 4041, "host-ts": 4041}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:2e60f807-1de0-44b3-a403-c38a2bf75bf2:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:2e60f807-1de0-44b3-a403-c38a2bf75bf2:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 32f59666 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 32f59666 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 32f59666 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3953 (Sat Oct 13 09:41:30 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3953 (Sat Oct 13 09:41:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5300571d", "local_conf_timestamp": 3953, "host-ts": 3953}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3960 (Sat Oct 13 09:41:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3960 (Sat Oct 13 09:41:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4856fb39", "local_conf_timestamp": 3960, "host-ts": 3960}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2eef2c49-c439-4c49-807a-fd834697673e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2eef2c49-c439-4c49-807a-fd834697673e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 34069ac8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 34069ac8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:470d81cc-1437-4e1c-bf28-bfa1eee53b5b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:470d81cc-1437-4e1c-bf28-bfa1eee53b5b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3517905c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3517905c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3517905c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3963 (Sat Oct 13 09:41:40 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3963 (Sat Oct 13 09:41:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecd8911d", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3970 (Sat Oct 13 09:41:47 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3970 (Sat Oct 13 09:41:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4c10043c", "local_conf_timestamp": 3970, "host-ts": 3970}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:55f1931e-70e2-4559-adfa-0e051770ff4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:55f1931e-70e2-4559-adfa-0e051770ff4c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3c8217ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3c8217ae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3c8217ae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3984 (Sat Oct 13 09:42:01 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3984 (Sat Oct 13 09:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "83d92228", "local_conf_timestamp": 3984, "host-ts": 3984}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3980 (Sat Oct 13 09:41:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3980 (Sat Oct 13 09:41:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73f7010f", "local_conf_timestamp": 3980, "host-ts": 3980}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:509e07e6-046d-4bac-8891-edbe344d97ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:509e07e6-046d-4bac-8891-edbe344d97ca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 40557470 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 40557470 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 40557470 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3984 (Sat Oct 13 09:42:01 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3984 (Sat Oct 13 09:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "83d92228", "local_conf_timestamp": 3984, "host-ts": 3984}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3980 (Sat Oct 13 09:41:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3980 (Sat Oct 13 09:41:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73f7010f", "local_conf_timestamp": 3980, "host-ts": 3980}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aa6fc850-c84e-46cf-bc99-6a9be33616b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aa6fc850-c84e-46cf-bc99-6a9be33616b2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 43581268 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 43581268 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 43581268 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3994 (Sat Oct 13 09:42:11 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Sat Oct 13 09:42:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3990 (Sat Oct 13 09:42:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3990 (Sat Oct 13 09:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "008ccb92", "local_conf_timestamp": 3990, "host-ts": 3990}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7fcd8428-6744-4b55-b56f-f1dc619b66f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7fcd8428-6744-4b55-b56f-f1dc619b66f7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4637f4f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4637f4f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4637f4f8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3994 (Sat Oct 13 09:42:11 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Sat Oct 13 09:42:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3990 (Sat Oct 13 09:42:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3990 (Sat Oct 13 09:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "008ccb92", "local_conf_timestamp": 3990, "host-ts": 3990}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:45cd725b-a4ba-47f5-be0f-38abdf20e35b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:45cd725b-a4ba-47f5-be0f-38abdf20e35b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4951e252 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4951e252 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4951e252 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4004 (Sat Oct 13 09:42:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4004 (Sat Oct 13 09:42:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3456cd63", "local_conf_timestamp": 4004, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4000 (Sat Oct 13 09:42:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4000 (Sat Oct 13 09:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "093eb926", "local_conf_timestamp": 4000, "host-ts": 4000}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d4a9e5e7-3bdf-4f52-9f4a-491cad5dee66:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d4a9e5e7-3bdf-4f52-9f4a-491cad5dee66:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4c35b660 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4c35b660 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4c35b660 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4004 (Sat Oct 13 09:42:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4004 (Sat Oct 13 09:42:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3456cd63", "local_conf_timestamp": 4004, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4000 (Sat Oct 13 09:42:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4000 (Sat Oct 13 09:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "093eb926", "local_conf_timestamp": 4000, "host-ts": 4000}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:132ae362-3d39-4c9a-abd7-55f4e8a95349:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:132ae362-3d39-4c9a-abd7-55f4e8a95349:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4f123fa2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4f123fa2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4f123fa2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4014 (Sat Oct 13 09:42:31 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4014 (Sat Oct 13 09:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9a65f3a1", "local_conf_timestamp": 4014, "host-ts": 4014}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4010 (Sat Oct 13 09:42:27 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4010 (Sat Oct 13 09:42:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be5c9ead", "local_conf_timestamp": 4010, "host-ts": 4010}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e5c3ff75-7c6d-4b8f-8467-06c1a7e17ecd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e5c3ff75-7c6d-4b8f-8467-06c1a7e17ecd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 523a6d8a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 523a6d8a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 523a6d8a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4014 (Sat Oct 13 09:42:31 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4014 (Sat Oct 13 09:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9a65f3a1", "local_conf_timestamp": 4014, "host-ts": 4014}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4010 (Sat Oct 13 09:42:27 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4010 (Sat Oct 13 09:42:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "be5c9ead", "local_conf_timestamp": 4010, "host-ts": 4010}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c48c94d4-c995-4b70-9bc0-192cf9ecbc50:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c48c94d4-c995-4b70-9bc0-192cf9ecbc50:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 553c4daa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 553c4daa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 553c4daa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4024 (Sat Oct 13 09:42:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4024 (Sat Oct 13 09:42:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b341b6a6", "local_conf_timestamp": 4024, "host-ts": 4024}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4020 (Sat Oct 13 09:42:37 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4020 (Sat Oct 13 09:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b2979fa2", "local_conf_timestamp": 4020, "host-ts": 4020}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f81612e6-7892-40b4-b170-de14d969e414:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f81612e6-7892-40b4-b170-de14d969e414:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 584aaff0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 584aaff0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 584aaff0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4024 (Sat Oct 13 09:42:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4024 (Sat Oct 13 09:42:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b341b6a6", "local_conf_timestamp": 4024, "host-ts": 4024}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4020 (Sat Oct 13 09:42:37 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4020 (Sat Oct 13 09:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b2979fa2", "local_conf_timestamp": 4020, "host-ts": 4020}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fa20f926-8b1c-4299-906b-ebd851133cb9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fa20f926-8b1c-4299-906b-ebd851133cb9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5b49bb56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5b49bb56 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5b49bb56 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4034 (Sat Oct 13 09:42:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4034 (Sat Oct 13 09:42:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "35aa34ac", "local_conf_timestamp": 4034, "host-ts": 4034}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4030 (Sat Oct 13 09:42:47 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4031 (Sat Oct 13 09:42:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1d65031", "local_conf_timestamp": 4031, "host-ts": 4030}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:702bdfca-69b1-4e2a-ab38-879b0e674c97:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:702bdfca-69b1-4e2a-ab38-879b0e674c97:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5c53f7aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5c53f7aa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5c53f7aa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4034 (Sat Oct 13 09:42:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4034 (Sat Oct 13 09:42:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "35aa34ac", "local_conf_timestamp": 4034, "host-ts": 4034}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4030 (Sat Oct 13 09:42:47 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4031 (Sat Oct 13 09:42:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c1d65031", "local_conf_timestamp": 4031, "host-ts": 4030}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2290d4e9-df48-4897-b1cf-b5046621634f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2290d4e9-df48-4897-b1cf-b5046621634f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 634fb012 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 634fb012 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 634fb012 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4044 (Sat Oct 13 09:43:01 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4044 (Sat Oct 13 09:43:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "2d6a3e8b", "local_conf_timestamp": 4044, "host-ts": 4044}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4041 (Sat Oct 13 09:42:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4041 (Sat Oct 13 09:42:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1b9bb73", "local_conf_timestamp": 4041, "host-ts": 4041}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/398/ Build Number: 398 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #395 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #396 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #397 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #398 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/399/ Build Number: 399 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #395 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #396 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #397 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #398 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #399 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/400/ Build Number: 400 Build Status: Fixed Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #382 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #383 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #384 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #385 [Eitan Raviv] networking: retrieve ipv6 of lago engine VM Changes for Build #386 [Eitan Raviv] networking: connect NFS to engine over IPv6 [Daniel Belenky] standard-pipelines: add kubevirt/common-templates Changes for Build #387 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #388 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #389 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #390 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #391 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #392 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #393 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #394 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #395 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #396 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #397 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #398 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #399 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #400 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- All tests passed
participants (1)
-
jenkins＠jenkins.phx.ovirt.org