[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 381 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/381/ Build Number: 381 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #351 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #352 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #353 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #354 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #355 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #356 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #357 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #358 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #359 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #360 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #361 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #362 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-setup-lib: drop 3.6 [Sandro Bonazzola] ovirt-host: drop 3.6 and 4.1 Changes for Build #363 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #364 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #365 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" [Sandro Bonazzola] ovirt-reports: drop 3.6 jobs [Sandro Bonazzola] ovirt-host-deploy: drop 3.6 jobs Changes for Build #366 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #367 [Miguel Duarte Barroso] Revert "Repos: add python2-ovsdbapp to the reposync config" Changes for Build #368 [Gal Ben Haim] Remove stale url to the internal repo [Greg Sheremeta] add ovirt-web-ui to stdci v2 Changes for Build #369 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #370 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #371 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #372 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #373 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #374 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #375 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #376 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #377 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #378 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #379 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #380 [Gal Ben Haim] Remove stale url to the internal repo Changes for Build #381 [Gal Ben Haim] Remove stale url to the internal repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running a1d1682e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Mon Oct 8 01:43:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4198 (Mon Oct 8 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3ce78604", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Mon Oct 8 01:43:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4206 (Mon Oct 8 01:43:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a30622a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command a30622a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a44300ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Mon Oct 8 01:43:56 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4208 (Mon Oct 8 01:43:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5b00040", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Mon Oct 8 01:44:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4216 (Mon Oct 8 01:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "60ac78cd", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ab57ee0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ad2bd33a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b483bd00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4239 (Mon Oct 8 01:44:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4240 (Mon Oct 8 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "62e27ecd", "local_conf_timestamp": 4240, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Mon Oct 8 01:44:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4237 (Mon Oct 8 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "681b83c9", "local_conf_timestamp": 4237, "host-ts": 4236}, "global_maintenance": false} lago.ssh: DEBUG: start task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b7b73a60 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false} lago.ssh: DEBUG: start task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running baa81564 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false} lago.ssh: DEBUG: start task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running be1b6e9e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false} lago.ssh: DEBUG: start task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c0fcb4ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false} lago.ssh: DEBUG: start task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c4258ed2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false} lago.ssh: DEBUG: start task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c823eace on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false} lago.ssh: DEBUG: start task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb0bd06c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false} lago.ssh: DEBUG: start task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cde223cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false} lago.ssh: DEBUG: start task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d0baaae2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4290 (Mon Oct 8 01:45:19 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4290 (Mon Oct 8 01:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "62b65a31", "local_conf_timestamp": 4290, "host-ts": 4290}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4286 (Mon Oct 8 01:45:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4286 (Mon Oct 8 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:8e6cfd85-fadd-4c09-b52b-ea235fad9164:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running a1d1682e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command a1d1682e on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Mon Oct 8 01:43:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4198 (Mon Oct 8 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3ce78604", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Mon Oct 8 01:43:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4206 (Mon Oct 8 01:43:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "64ea87c8", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5708fd23-ba2d-4309-a0ce-9357359543a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a30622a2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command a30622a2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:67ad3d76-e98a-4499-97a4-49e4d327cbc9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a44300ae on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a44300ae on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Mon Oct 8 01:43:56 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4208 (Mon Oct 8 01:43:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b5b00040", "local_conf_timestamp": 4208, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Mon Oct 8 01:44:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4216 (Mon Oct 8 01:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "60ac78cd", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:33612a9b-a079-400a-90cd-d82c2e45b62a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ab57ee0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ab57ee0e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:68d64fe3-62c6-4953-ad67-f1ffccc23943:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ad2bd33a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ad2bd33a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Mon Oct 8 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4230 (Mon Oct 8 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e84d531d", "local_conf_timestamp": 4230, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Mon Oct 8 01:44:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4226 (Mon Oct 8 01:44:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1b5a4c5a", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3b29029e-6aed-4c70-9cf3-b0e037b37e4e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b483bd00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b483bd00 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4239 (Mon Oct 8 01:44:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4240 (Mon Oct 8 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "62e27ecd", "local_conf_timestamp": 4240, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Mon Oct 8 01:44:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4237 (Mon Oct 8 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "681b83c9", "local_conf_timestamp": 4237, "host-ts": 4236}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5ba2509d-768e-4570-ac52-270f0e4e5ee4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b7b73a60 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b7b73a60 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:41d93871-3bc6-4e72-9c4b-1ca9d0acef68:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running baa81564 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command baa81564 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Mon Oct 8 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4251 (Mon Oct 8 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ec043c9a", "local_conf_timestamp": 4251, "host-ts": 4250}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Mon Oct 8 01:44:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Mon Oct 8 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "02cc4e44", "local_conf_timestamp": 4246, "host-ts": 4246}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5eadd97d-f155-4167-964d-43c591453f51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running be1b6e9e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command be1b6e9e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9b935f9f-dd74-46f5-bf43-136d11ceb7d0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c0fcb4ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c0fcb4ba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Mon Oct 8 01:44:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4260 (Mon Oct 8 01:44:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b227490b", "local_conf_timestamp": 4260, "host-ts": 4260}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Mon Oct 8 01:44:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4256 (Mon Oct 8 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "068ab141", "local_conf_timestamp": 4256, "host-ts": 4256}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6379f4f4-dab0-4167-9c2e-9c93a10523e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c4258ed2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c4258ed2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cf3a56cb-3e35-4f59-9611-fd585210c1cf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c823eace on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c823eace on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Mon Oct 8 01:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4271 (Mon Oct 8 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6b13475f", "local_conf_timestamp": 4271, "host-ts": 4270}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Mon Oct 8 01:44:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4266 (Mon Oct 8 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b96568c0", "local_conf_timestamp": 4266, "host-ts": 4266}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0d0db340-e017-4d64-b1ac-89d0a04bcf93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb0bd06c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cb0bd06c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ac349683-8c23-4dc6-bf7f-5c83686aef05:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cde223cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cde223cc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4280 (Mon Oct 8 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4280 (Mon Oct 8 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1ab6b87d", "local_conf_timestamp": 4280, "host-ts": 4280}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4276 (Mon Oct 8 01:45:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4277 (Mon Oct 8 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ca24a753", "local_conf_timestamp": 4277, "host-ts": 4276}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aacb982d-3477-40e5-a2c0-673d4b3e9a1a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d0baaae2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d0baaae2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4290 (Mon Oct 8 01:45:19 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4290 (Mon Oct 8 01:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "62b65a31", "local_conf_timestamp": 4290, "host-ts": 4290}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4286 (Mon Oct 8 01:45:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4286 (Mon Oct 8 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'