[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 352 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/352/ Build Number: 352 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4174 (Fri Sep 28 09:43:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4174 (Fri Sep 28 09:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Fri Sep 28 09:43:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4181 (Fri Sep 28 09:43:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false} lago.ssh: DEBUG: start task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fd299584 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command fd299584 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fe44418a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Fri Sep 28 09:43:38 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4185 (Fri Sep 28 09:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b51bae61", "local_conf_timestamp": 4185, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 05507b9c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Fri Sep 28 09:43:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4195 (Fri Sep 28 09:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c5979ca", "local_conf_timestamp": 4195, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false} lago.ssh: DEBUG: start task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 08641ae6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4201 (Fri Sep 28 09:43:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4201 (Fri Sep 28 09:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false} lago.ssh: DEBUG: start task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0bba372a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0ed6b14a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Fri Sep 28 09:44:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Fri Sep 28 09:44:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false} lago.ssh: DEBUG: start task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 11d1e234 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 15a747aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false} lago.ssh: DEBUG: start task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 18a081ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false} lago.ssh: DEBUG: start task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1ba6f34e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false} lago.ssh: DEBUG: start task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1f62be8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false} lago.ssh: DEBUG: start task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 22e5be24 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false} lago.ssh: DEBUG: start task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 25f6d062 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false} lago.ssh: DEBUG: start task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 270d7942 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false} lago.ssh: DEBUG: start task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2e308b56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Fri Sep 28 09:45:01 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4268 (Fri Sep 28 09:45:01 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "d6243e92", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4261 (Fri Sep 28 09:44:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4261 (Fri Sep 28 09:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2403aff7", "local_conf_timestamp": 4261, "host-ts": 4261}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:2c3326a9-a2b1-4254-8baa-b9a39982382c:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command fc0b0fb6 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4174 (Fri Sep 28 09:43:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4174 (Fri Sep 28 09:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Fri Sep 28 09:43:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4181 (Fri Sep 28 09:43:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b183d660", "local_conf_timestamp": 4181, "host-ts": 4181}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bda40c48-f97b-4a13-9a92-f9e20caeaefb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fd299584 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command fd299584 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3737861b-5d6d-428e-95cc-e16c0bdcd9ff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fe44418a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fe44418a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Fri Sep 28 09:43:38 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4185 (Fri Sep 28 09:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b51bae61", "local_conf_timestamp": 4185, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:49e4540d-aae9-44f1-b565-4663bcfd6e11:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 05507b9c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 05507b9c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Fri Sep 28 09:43:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4195 (Fri Sep 28 09:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8c5979ca", "local_conf_timestamp": 4195, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Fri Sep 28 09:43:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4191 (Fri Sep 28 09:43:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5c52965", "local_conf_timestamp": 4191, "host-ts": 4191}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2cb185cd-31ee-4406-98d8-a6b674ab7e15:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 08641ae6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 08641ae6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4201 (Fri Sep 28 09:43:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4201 (Fri Sep 28 09:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "743acd40", "local_conf_timestamp": 4201, "host-ts": 4201}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e267c85e-5dfe-45ff-94e2-2c3e5b9b0fdb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0bba372a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0bba372a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Fri Sep 28 09:43:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Fri Sep 28 09:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:45b43398-ee47-42ab-8c56-fc3dee4b44bb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0ed6b14a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0ed6b14a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Fri Sep 28 09:44:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Fri Sep 28 09:44:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4211 (Fri Sep 28 09:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4211 (Fri Sep 28 09:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "707c3245", "local_conf_timestamp": 4211, "host-ts": 4211}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fa78761a-3c2f-4f4a-8580-a79eab8f11d2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 11d1e234 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 11d1e234 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c6ae7ee6-73ba-4fc4-9cd5-7c87d738e328:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 15a747aa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 15a747aa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Sep 28 09:44:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4226 (Fri Sep 28 09:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47108073", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4221 (Fri Sep 28 09:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4221 (Fri Sep 28 09:44:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7cb7334a", "local_conf_timestamp": 4221, "host-ts": 4221}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e92bcf42-8d0d-4331-a863-be13c5108ff5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 18a081ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 18a081ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:59aac654-4e35-41c6-a9cf-2958dbbc71d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1ba6f34e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1ba6f34e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 09:44:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4231 (Fri Sep 28 09:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4231 (Fri Sep 28 09:44:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "78f1cc4f", "local_conf_timestamp": 4231, "host-ts": 4231}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a27b350e-7c60-4a55-8ec6-4199df031738:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1f62be8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1f62be8c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8ed5059a-73af-4eac-ba9c-6bc907a1c685:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 22e5be24 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 22e5be24 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 09:44:40 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 09:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4241 (Fri Sep 28 09:44:33 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4241 (Fri Sep 28 09:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d605e9da", "local_conf_timestamp": 4241, "host-ts": 4241}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:809cf4bd-1b5e-46e0-8941-034f269449ba:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 25f6d062 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 25f6d062 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ed8e5dca-e55a-42cc-a204-8a102c26fa56:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 270d7942 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 270d7942 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 09:44:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Fri Sep 28 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5d8727c8", "local_conf_timestamp": 4258, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4251 (Fri Sep 28 09:44:44 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4251 (Fri Sep 28 09:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d24316df", "local_conf_timestamp": 4251, "host-ts": 4251}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b06e9061-2978-4781-815e-9441a29f750f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2e308b56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2e308b56 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Fri Sep 28 09:45:01 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4268 (Fri Sep 28 09:45:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "d6243e92", "local_conf_timestamp": 4268, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4261 (Fri Sep 28 09:44:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4261 (Fri Sep 28 09:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2403aff7", "local_conf_timestamp": 4261, "host-ts": 4261}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'