[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 357 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/357/ Build Number: 357 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #347 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #348 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #349 [Dafna Ron] ovirt-4.2: block external links from running in ost Changes for Build #350 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 0350105a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4159 (Sun Sep 30 01:41:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4159 (Sun Sep 30 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false} lago.ssh: DEBUG: start task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 04612d44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 04612d44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 05749c84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false} lago.ssh: DEBUG: start task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0c910dcc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sun Sep 30 01:42:02 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4175 (Sun Sep 30 01:42:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d16ed36", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false} lago.ssh: DEBUG: start task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0f9a3bce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false} lago.ssh: DEBUG: start task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 127d027c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false} lago.ssh: DEBUG: start task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 15ef7d7c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Sep 30 01:42:22 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4196 (Sun Sep 30 01:42:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4189 (Sun Sep 30 01:42:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4189 (Sun Sep 30 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f350724e", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false} lago.ssh: DEBUG: start task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 18f6d48e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false} lago.ssh: DEBUG: start task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1f13ce1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 223f3bee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false} lago.ssh: DEBUG: start task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 25806706 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false} lago.ssh: DEBUG: start task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 28a3f2cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false} lago.ssh: DEBUG: start task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 33ce4562 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Sun Sep 30 01:43:12 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4245 (Sun Sep 30 01:43:12 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "df015cca", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Sun Sep 30 01:43:07 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4240 (Sun Sep 30 01:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f6f1d9e6", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:fa05feae-9b57-4b45-8477-fe6c09be8a5e:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 0350105a on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 0350105a on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4159 (Sun Sep 30 01:41:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4159 (Sun Sep 30 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b307bcef", "local_conf_timestamp": 4159, "host-ts": 4159}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0520e71d-5c70-49d4-92d5-8af4cc9e0b6a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 04612d44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 04612d44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ac915f65-3794-4c92-a1a6-88e5a0f29628:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 05749c84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 05749c84 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Sun Sep 30 01:41:52 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4165 (Sun Sep 30 01:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "821efa25", "local_conf_timestamp": 4165, "host-ts": 4165}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e71b59ea-9b6b-41a6-97b2-81f86ab70988:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0c910dcc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0c910dcc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sun Sep 30 01:42:02 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4175 (Sun Sep 30 01:42:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d16ed36", "local_conf_timestamp": 4175, "host-ts": 4175}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Sun Sep 30 01:41:56 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4169 (Sun Sep 30 01:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bfccbde0", "local_conf_timestamp": 4169, "host-ts": 4169}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4d645538-8bc4-426b-9ae5-8dbb60de34e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0f9a3bce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0f9a3bce on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b80d5eb5-e522-470b-b51c-df0b47c1033d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 127d027c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 127d027c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Sep 30 01:42:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4186 (Sun Sep 30 01:42:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4b016144", "local_conf_timestamp": 4186, "host-ts": 4185}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Sun Sep 30 01:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4179 (Sun Sep 30 01:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb8a42e5", "local_conf_timestamp": 4179, "host-ts": 4179}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76fdba44-75e2-4d5f-b2bc-6afb1eb4e82d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 15ef7d7c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 15ef7d7c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Sep 30 01:42:22 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4196 (Sun Sep 30 01:42:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e5325f86", "local_conf_timestamp": 4196, "host-ts": 4195}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4189 (Sun Sep 30 01:42:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4189 (Sun Sep 30 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f350724e", "local_conf_timestamp": 4189, "host-ts": 4189}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3c0863a9-64fe-4012-bfd3-fbc237f0828f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 18f6d48e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 18f6d48e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:379a74fb-6053-47ff-8b0a-fd8e77ae329b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1c0f8f08 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 30 01:42:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Sun Sep 30 01:42:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Sun Sep 30 01:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4199 (Sun Sep 30 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7168d4b", "local_conf_timestamp": 4199, "host-ts": 4199}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5282fc73-94ce-4287-987c-03fb1de07d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1f13ce1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1f13ce1c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:55ed5198-5ad8-4a43-8884-10be36008c45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 223f3bee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 223f3bee on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 30 01:42:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Sun Sep 30 01:42:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Sun Sep 30 01:42:36 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4209 (Sun Sep 30 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2f08478", "local_conf_timestamp": 4209, "host-ts": 4209}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:01e9b2e4-afa2-48d2-9870-ae1065688e98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 25806706 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 25806706 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:284ecda4-12a7-408e-a160-d7c5a55d51a3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 28a3f2cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 28a3f2cc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 30 01:42:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 30 01:42:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Sun Sep 30 01:42:46 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4220 (Sun Sep 30 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a447901a", "local_conf_timestamp": 4220, "host-ts": 4219}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dfa452ce-5881-42c2-bebb-e4a153cdd1db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2bb88ff4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5ad318d5-376e-4b34-8b11-6e69e1972fe4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2ccc4d9a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 30 01:43:02 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Sun Sep 30 01:43:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e9fc34e2", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Sun Sep 30 01:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4230 (Sun Sep 30 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "560c0447", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:772f271c-57a8-41b3-ad7d-074b24c1ce3d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 33ce4562 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 33ce4562 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Sun Sep 30 01:43:12 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4245 (Sun Sep 30 01:43:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "df015cca", "local_conf_timestamp": 4245, "host-ts": 4245}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Sun Sep 30 01:43:07 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4240 (Sun Sep 30 01:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f6f1d9e6", "local_conf_timestamp": 4240, "host-ts": 4240}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'