[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 414 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/414/ Build Number: 414 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 3214cc76 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3350e1ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 3350e1ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 34672c44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3ca0a110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3fec8186 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 42e03626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 45c9751e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 48cfa1ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false} lago.ssh: DEBUG: start task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4c0c0892 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false} lago.ssh: DEBUG: start task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4ef383be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false} lago.ssh: DEBUG: start task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 51f18f0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false} lago.ssh: DEBUG: start task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5530d7c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false} lago.ssh: DEBUG: start task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5843316c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false} lago.ssh: DEBUG: start task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false} lago.ssh: DEBUG: start task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5c5bb24c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false} lago.ssh: DEBUG: start task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 63658f04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4011 (Fri Oct 19 01:43:26 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4011 (Fri Oct 19 01:43:26 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bfe68fc7", "local_conf_timestamp": 4011, "host-ts": 4011}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Fri Oct 19 01:43:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Fri Oct 19 01:43:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96c56b45", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 3214cc76 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3350e1ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 3350e1ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 34672c44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3ca0a110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3fec8186 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 42e03626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 45c9751e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 48cfa1ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4c0c0892 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4ef383be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 51f18f0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5530d7c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5843316c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5c5bb24c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 63658f04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4011 (Fri Oct 19 01:43:26 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4011 (Fri Oct 19 01:43:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bfe68fc7", "local_conf_timestamp": 4011, "host-ts": 4011}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Fri Oct 19 01:43:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Fri Oct 19 01:43:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96c56b45", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'