[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 353 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/353/ Build Number: 353 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 64ce80d0 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Fri Sep 28 17:46:04 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4206 (Fri Sep 28 17:46:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bf68f918", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Fri Sep 28 17:46:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4207 (Fri Sep 28 17:46:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f933974e", "local_conf_timestamp": 4207, "host-ts": 4207}, "global_maintenance": false} lago.ssh: DEBUG: start task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 66124940 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 66124940 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 67382574 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Fri Sep 28 17:46:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "85a6ce38", "local_conf_timestamp": 4217, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false} lago.ssh: DEBUG: start task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6ea8843e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Fri Sep 28 17:46:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4226 (Fri Sep 28 17:46:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e166f82", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false} lago.ssh: DEBUG: start task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71efd110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false} lago.ssh: DEBUG: start task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 756a0e14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false} lago.ssh: DEBUG: start task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 78530158 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false} lago.ssh: DEBUG: start task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7b5a7214 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false} lago.ssh: DEBUG: start task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7e568156 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false} lago.ssh: DEBUG: start task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 815cfd30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false} lago.ssh: DEBUG: start task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 84889992 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8791ddf6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8db8ea76 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false} lago.ssh: DEBUG: start task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9092e81e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false} lago.ssh: DEBUG: start task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:a4139e28-20a1-46aa-88cc-9877e94ad1db:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 64ce80d0 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 64ce80d0 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Fri Sep 28 17:46:04 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4206 (Fri Sep 28 17:46:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bf68f918", "local_conf_timestamp": 4206, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Fri Sep 28 17:46:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4207 (Fri Sep 28 17:46:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f933974e", "local_conf_timestamp": 4207, "host-ts": 4207}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3f085203-5a68-4e07-a93a-0ac319d49203:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 66124940 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 66124940 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9cad80ae-9b4f-44d4-955a-10acfd3cf51e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 67382574 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 67382574 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Fri Sep 28 17:46:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "85a6ce38", "local_conf_timestamp": 4217, "host-ts": 4216}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0eae7182-4081-4b16-a237-d059ec2c4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6ea8843e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6ea8843e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Fri Sep 28 17:46:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4226 (Fri Sep 28 17:46:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6e166f82", "local_conf_timestamp": 4226, "host-ts": 4226}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Fri Sep 28 17:46:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4217 (Fri Sep 28 17:46:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd75684b", "local_conf_timestamp": 4217, "host-ts": 4217}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:454b4892-6a47-49df-abe6-81cb452d9845:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71efd110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71efd110 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:54561bf9-eddd-451f-832a-96b58b073e1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 756a0e14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 756a0e14 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Fri Sep 28 17:46:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9e248e27", "local_conf_timestamp": 4237, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Fri Sep 28 17:46:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4227 (Fri Sep 28 17:46:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f1be6944", "local_conf_timestamp": 4227, "host-ts": 4227}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a6da20a9-6759-4bb7-9083-7b4a9fa67797:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 78530158 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 78530158 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6be4f572-98ed-47ec-a099-5183229f4273:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7b5a7214 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7b5a7214 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4247 (Fri Sep 28 17:46:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4247 (Fri Sep 28 17:46:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f70c8295", "local_conf_timestamp": 4247, "host-ts": 4247}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Fri Sep 28 17:46:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4237 (Fri Sep 28 17:46:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "82c5a3d9", "local_conf_timestamp": 4237, "host-ts": 4237}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76ffd1f8-1ecd-4785-86ba-545051f32043:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7e568156 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7e568156 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2213ef05-6460-49ae-90de-c2bb2b8c1823:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 815cfd30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 815cfd30 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "593fbc57", "local_conf_timestamp": 4257, "host-ts": 4257}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4257 (Fri Sep 28 17:46:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4257 (Fri Sep 28 17:46:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9b53a1c7", "local_conf_timestamp": 4257, "host-ts": 4257}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:586c072a-a63a-465c-a7fa-86b80daa9a6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 84889992 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 84889992 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c2e7382f-516f-428c-a813-c2e8839f95cb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8791ddf6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8791ddf6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4268 (Fri Sep 28 17:47:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e0a4e4c1", "local_conf_timestamp": 4268, "host-ts": 4267}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9789f306-7b91-42f3-a307-6c16449eb492:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8a9dbac4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3fa77d29-5fc0-480a-812d-cfab213ed8a1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8db8ea76 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8db8ea76 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4277 (Fri Sep 28 17:47:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4277 (Fri Sep 28 17:47:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "de28c792", "local_conf_timestamp": 4277, "host-ts": 4277}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4267 (Fri Sep 28 17:47:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4267 (Fri Sep 28 17:47:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "24bc7846", "local_conf_timestamp": 4267, "host-ts": 4267}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:80d754ee-a5b3-4b77-8f9c-ebf8dd8fe841:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9092e81e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9092e81e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ceef6714-59a8-4f8a-9353-e28d7d408f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 91a2ccb0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b4e7f142", "local_conf_timestamp": 4287, "host-ts": 4287}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4287 (Fri Sep 28 17:47:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4287 (Fri Sep 28 17:47:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c402176d", "local_conf_timestamp": 4287, "host-ts": 4287}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'