[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 411 - Failure!
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/411/ Build Number: 411 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 5e792150 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3933 (Thu Oct 18 01:44:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3933 (Thu Oct 18 01:44:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7c0dd5c", "local_conf_timestamp": 3933, "host-ts": 3933}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5f815572 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 5f815572 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 60b78506 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3943 (Thu Oct 18 01:44:38 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3943 (Thu Oct 18 01:44:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8f671722", "local_conf_timestamp": 3943, "host-ts": 3943}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 67d368b4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3953 (Thu Oct 18 01:44:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3953 (Thu Oct 18 01:44:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "706f0031", "local_conf_timestamp": 3953, "host-ts": 3953}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Thu Oct 18 01:44:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3945 (Thu Oct 18 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6bf19f06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6ef7539e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71ed0dfa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false} lago.ssh: DEBUG: start task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 74df0478 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false} lago.ssh: DEBUG: start task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 77d8a918 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false} lago.ssh: DEBUG: start task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7ac56292 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false} lago.ssh: DEBUG: start task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7db577e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false} lago.ssh: DEBUG: start task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 81550626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in _wait_for_engine_migration testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 246, in assert_equals_within time.sleep(3) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 5e792150 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3933 (Thu Oct 18 01:44:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3933 (Thu Oct 18 01:44:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7c0dd5c", "local_conf_timestamp": 3933, "host-ts": 3933}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5f815572 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 5f815572 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 60b78506 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3943 (Thu Oct 18 01:44:38 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3943 (Thu Oct 18 01:44:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8f671722", "local_conf_timestamp": 3943, "host-ts": 3943}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 67d368b4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3953 (Thu Oct 18 01:44:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3953 (Thu Oct 18 01:44:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "706f0031", "local_conf_timestamp": 3953, "host-ts": 3953}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Thu Oct 18 01:44:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3945 (Thu Oct 18 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6bf19f06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6ef7539e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71ed0dfa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 74df0478 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 77d8a918 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7ac56292 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7db577e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 81550626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/412/ Build Number: 412 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 65, in local_maintenance lambda: host_service.get().status == types.HostStatus.MAINTENANCE or File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/413/ Build Number: 413 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/414/ Build Number: 414 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 3214cc76 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3350e1ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 3350e1ba on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 34672c44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3ca0a110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3fec8186 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 42e03626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 45c9751e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 48cfa1ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false} lago.ssh: DEBUG: start task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4c0c0892 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false} lago.ssh: DEBUG: start task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4ef383be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false} lago.ssh: DEBUG: start task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 51f18f0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false} lago.ssh: DEBUG: start task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5530d7c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false} lago.ssh: DEBUG: start task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5843316c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false} lago.ssh: DEBUG: start task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false} lago.ssh: DEBUG: start task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5c5bb24c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false} lago.ssh: DEBUG: start task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 63658f04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4011 (Fri Oct 19 01:43:26 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4011 (Fri Oct 19 01:43:26 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bfe68fc7", "local_conf_timestamp": 4011, "host-ts": 4011}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Fri Oct 19 01:43:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Fri Oct 19 01:43:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96c56b45", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:853aed7c-3f5e-4c56-936d-c49375a281e0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 3214cc76 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 3214cc76 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76793f91-9312-4d82-9484-79c47ecf0575:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3350e1ba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 3350e1ba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e3b452b4-38ca-4fb5-9022-26b81bc27fec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 34672c44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 34672c44 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3931 (Fri Oct 19 01:42:07 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3932 (Fri Oct 19 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d7236afb", "local_conf_timestamp": 3932, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Fri Oct 19 01:42:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3936 (Fri Oct 19 01:42:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "432cf1cf", "local_conf_timestamp": 3936, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:48978fe3-2a99-4199-8aaa-caeb6f3b2718:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3ca0a110 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3ca0a110 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3fa57fca-0b17-45a4-b2ce-67b3ce3f167e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3fec8186 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3fec8186 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Fri Oct 19 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3953 (Fri Oct 19 01:42:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba91ce10", "local_conf_timestamp": 3953, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Fri Oct 19 01:42:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3945 (Fri Oct 19 01:42:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d03378f7-559d-4336-9282-4fc55b8e930b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 42e03626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 42e03626 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:94c53e06-4c22-489b-8f39-47ddb8bb0487:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 45c9751e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 45c9751e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3961 (Fri Oct 19 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3961 (Fri Oct 19 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7dbbea3b", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Fri Oct 19 01:42:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3955 (Fri Oct 19 01:42:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b48e97f3", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3e98498c-744e-4467-b3cb-78b502afd4d8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 48cfa1ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 48cfa1ac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9fbafb6e-6cfe-4297-b46e-afbf0cac9c94:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4c0c0892 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4c0c0892 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3971 (Fri Oct 19 01:42:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3971 (Fri Oct 19 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d388d4f9", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Fri Oct 19 01:42:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Fri Oct 19 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:313b7d90-13aa-421c-a7f6-2c8abfa72aef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4ef383be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4ef383be on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7d66f5ae-2b50-4273-baa2-6f23b383546d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 51f18f0c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 51f18f0c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3981 (Fri Oct 19 01:42:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3981 (Fri Oct 19 01:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5e3d85e2", "local_conf_timestamp": 3981, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Fri Oct 19 01:42:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Fri Oct 19 01:42:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e829d444-4b47-44ec-99df-6b75c145455c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5530d7c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5530d7c2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0fc3d77c-40e0-4414-888a-8bc86642df43:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5843316c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5843316c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3991 (Fri Oct 19 01:43:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3991 (Fri Oct 19 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f00ebb20", "local_conf_timestamp": 3991, "host-ts": 3991}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Fri Oct 19 01:43:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3985 (Fri Oct 19 01:43:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "30c0b444", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5c2cf252-38f8-4ac8-a38e-2e8940865829:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5b3ceb7e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fae39b3b-d79e-4c82-9742-e0157a59786c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5c5bb24c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5c5bb24c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4001 (Fri Oct 19 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4001 (Fri Oct 19 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac057a05", "local_conf_timestamp": 4001, "host-ts": 4001}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3995 (Fri Oct 19 01:43:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3995 (Fri Oct 19 01:43:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34864b41", "local_conf_timestamp": 3995, "host-ts": 3995}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aca2fe09-607c-475e-ab08-7120f3088337:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 63658f04 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 63658f04 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4011 (Fri Oct 19 01:43:26 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4011 (Fri Oct 19 01:43:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bfe68fc7", "local_conf_timestamp": 4011, "host-ts": 4011}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Fri Oct 19 01:43:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Fri Oct 19 01:43:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96c56b45", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/415/ Build Number: 415 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:228bc56c-b9a5-41a1-b7bd-211fd427aad2:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:228bc56c-b9a5-41a1-b7bd-211fd427aad2:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running f6b7afa2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f6b7afa2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command f6b7afa2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Fri Oct 19 09:39:58 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3952 (Fri Oct 19 09:39:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73e3e0ba", "local_conf_timestamp": 3952, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3952 (Fri Oct 19 09:39:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3952 (Fri Oct 19 09:39:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a45edd7b", "local_conf_timestamp": 3952, "host-ts": 3952}, "global_maintenance": false} lago.ssh: DEBUG: start task:aa7ada9c-88e7-4c94-9d52-0e7098bdde2e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aa7ada9c-88e7-4c94-9d52-0e7098bdde2e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f7c3039c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command f7c3039c on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:22bd49d5-3b06-4b82-8721-0eb1f3fa999d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:22bd49d5-3b06-4b82-8721-0eb1f3fa999d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f8d698c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f8d698c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f8d698c0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3961 (Fri Oct 19 09:40:08 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3961 (Fri Oct 19 09:40:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "55327700", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3962 (Fri Oct 19 09:40:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3962 (Fri Oct 19 09:40:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a895dc74", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false} lago.ssh: DEBUG: start task:90a8c70e-b52b-429d-ab49-7d9bba3b4ee8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:90a8c70e-b52b-429d-ab49-7d9bba3b4ee8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ffec19be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ffec19be on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ffec19be on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3971 (Fri Oct 19 09:40:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3971 (Fri Oct 19 09:40:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "aa3a6013", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3962 (Fri Oct 19 09:40:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3962 (Fri Oct 19 09:40:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a895dc74", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false} lago.ssh: DEBUG: start task:df586988-f3e3-4969-ad4b-4553b2cd4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:df586988-f3e3-4969-ad4b-4553b2cd4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 036ed360 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 036ed360 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 036ed360 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3981 (Fri Oct 19 09:40:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c734d458", "local_conf_timestamp": 3982, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3982 (Fri Oct 19 09:40:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e40913da", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false} lago.ssh: DEBUG: start task:c6a98e53-6162-4988-b384-bdaf300bcf80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c6a98e53-6162-4988-b384-bdaf300bcf80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 073e4d0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 073e4d0e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 073e4d0e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3981 (Fri Oct 19 09:40:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c734d458", "local_conf_timestamp": 3982, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3982 (Fri Oct 19 09:40:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e40913da", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false} lago.ssh: DEBUG: start task:b0c61fbc-9b93-48f4-a0c8-992fb51648f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b0c61fbc-9b93-48f4-a0c8-992fb51648f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0ac1b100 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0ac1b100 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0ac1b100 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3993 (Fri Oct 19 09:40:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7a8514cf", "local_conf_timestamp": 3993, "host-ts": 3992}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false} lago.ssh: DEBUG: start task:5b18421a-9f4b-49ae-a5ac-ee53aa378171:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5b18421a-9f4b-49ae-a5ac-ee53aa378171:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0dbcd15a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0dbcd15a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0dbcd15a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3993 (Fri Oct 19 09:40:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7a8514cf", "local_conf_timestamp": 3993, "host-ts": 3992}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false} lago.ssh: DEBUG: start task:e87ef80e-c58d-43b8-8f9f-f7e74dbb73aa:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e87ef80e-c58d-43b8-8f9f-f7e74dbb73aa:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 10bd01d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 10bd01d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 10bd01d6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "143ef5d0", "local_conf_timestamp": 4002, "host-ts": 4002}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false} lago.ssh: DEBUG: start task:72312e92-c5c7-420a-82e3-aacb317304d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:72312e92-c5c7-420a-82e3-aacb317304d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 13f2bfee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 13f2bfee on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 13f2bfee on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "143ef5d0", "local_conf_timestamp": 4002, "host-ts": 4002}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false} lago.ssh: DEBUG: start task:098baf60-62cf-4325-82da-fe6080f23166:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:098baf60-62cf-4325-82da-fe6080f23166:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 170833f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 170833f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 170833f8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4012 (Fri Oct 19 09:40:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4012 (Fri Oct 19 09:40:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba0dcb12", "local_conf_timestamp": 4012, "host-ts": 4012}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e9fd9e6b", "local_conf_timestamp": 4002, "host-ts": 4002}, "global_maintenance": false} lago.ssh: DEBUG: start task:385b4460-1bb6-4b3a-a4ae-108491ed06a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:385b4460-1bb6-4b3a-a4ae-108491ed06a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1a2d1dc8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1a2d1dc8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1a2d1dc8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4012 (Fri Oct 19 09:40:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4012 (Fri Oct 19 09:40:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba0dcb12", "local_conf_timestamp": 4012, "host-ts": 4012}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e9fd9e6b", "local_conf_timestamp": 4002, "host-ts": 4002}, "global_maintenance": false} lago.ssh: DEBUG: start task:9799da1c-ab86-4b99-aaa8-aed6e378cf71:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9799da1c-ab86-4b99-aaa8-aed6e378cf71:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1d12f4ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1d12f4ea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1d12f4ea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4022 (Fri Oct 19 09:41:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4023 (Fri Oct 19 09:41:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e42ebe83", "local_conf_timestamp": 4023, "host-ts": 4022}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4022 (Fri Oct 19 09:41:08 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4022 (Fri Oct 19 09:41:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5254b8ef", "local_conf_timestamp": 4022, "host-ts": 4022}, "global_maintenance": false} lago.ssh: DEBUG: start task:1e393081-d77e-44e6-9a11-393ef1c2681e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1e393081-d77e-44e6-9a11-393ef1c2681e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 208f99e8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 208f99e8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 208f99e8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4022 (Fri Oct 19 09:41:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4023 (Fri Oct 19 09:41:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e42ebe83", "local_conf_timestamp": 4023, "host-ts": 4022}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4022 (Fri Oct 19 09:41:08 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4022 (Fri Oct 19 09:41:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5254b8ef", "local_conf_timestamp": 4022, "host-ts": 4022}, "global_maintenance": false} lago.ssh: DEBUG: start task:fe84747a-4b28-44c3-91be-1cc097fa874e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fe84747a-4b28-44c3-91be-1cc097fa874e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2397ee42 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2397ee42 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2397ee42 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4033 (Fri Oct 19 09:41:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4033 (Fri Oct 19 09:41:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9bfb7d93", "local_conf_timestamp": 4033, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4032 (Fri Oct 19 09:41:19 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4032 (Fri Oct 19 09:41:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "561247ea", "local_conf_timestamp": 4032, "host-ts": 4032}, "global_maintenance": false} lago.ssh: DEBUG: start task:33fa4360-4885-4c5b-ae5e-7489c048b04b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:33fa4360-4885-4c5b-ae5e-7489c048b04b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 24b53866 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 24b53866 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 24b53866 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4033 (Fri Oct 19 09:41:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4033 (Fri Oct 19 09:41:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9bfb7d93", "local_conf_timestamp": 4033, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4032 (Fri Oct 19 09:41:19 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4032 (Fri Oct 19 09:41:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "561247ea", "local_conf_timestamp": 4032, "host-ts": 4032}, "global_maintenance": false} lago.ssh: DEBUG: start task:44aae76a-d180-4046-9599-f546858a588a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:44aae76a-d180-4046-9599-f546858a588a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2bcab28e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2bcab28e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2bcab28e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4043 (Fri Oct 19 09:41:30 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4043 (Fri Oct 19 09:41:30 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "f51c8fbd", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4042 (Fri Oct 19 09:41:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4042 (Fri Oct 19 09:41:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f91ead40", "local_conf_timestamp": 4042, "host-ts": 4042}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:228bc56c-b9a5-41a1-b7bd-211fd427aad2:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:228bc56c-b9a5-41a1-b7bd-211fd427aad2:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running f6b7afa2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f6b7afa2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command f6b7afa2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Fri Oct 19 09:39:58 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3952 (Fri Oct 19 09:39:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "73e3e0ba", "local_conf_timestamp": 3952, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3952 (Fri Oct 19 09:39:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3952 (Fri Oct 19 09:39:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a45edd7b", "local_conf_timestamp": 3952, "host-ts": 3952}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aa7ada9c-88e7-4c94-9d52-0e7098bdde2e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aa7ada9c-88e7-4c94-9d52-0e7098bdde2e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f7c3039c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command f7c3039c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:22bd49d5-3b06-4b82-8721-0eb1f3fa999d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:22bd49d5-3b06-4b82-8721-0eb1f3fa999d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f8d698c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f8d698c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f8d698c0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3961 (Fri Oct 19 09:40:08 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3961 (Fri Oct 19 09:40:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "55327700", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3962 (Fri Oct 19 09:40:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3962 (Fri Oct 19 09:40:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a895dc74", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:90a8c70e-b52b-429d-ab49-7d9bba3b4ee8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:90a8c70e-b52b-429d-ab49-7d9bba3b4ee8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ffec19be on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ffec19be on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ffec19be on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3971 (Fri Oct 19 09:40:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3971 (Fri Oct 19 09:40:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "aa3a6013", "local_conf_timestamp": 3971, "host-ts": 3971}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3962 (Fri Oct 19 09:40:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3962 (Fri Oct 19 09:40:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a895dc74", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:df586988-f3e3-4969-ad4b-4553b2cd4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:df586988-f3e3-4969-ad4b-4553b2cd4fb0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 036ed360 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 036ed360 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 036ed360 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3981 (Fri Oct 19 09:40:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c734d458", "local_conf_timestamp": 3982, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3982 (Fri Oct 19 09:40:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e40913da", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c6a98e53-6162-4988-b384-bdaf300bcf80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c6a98e53-6162-4988-b384-bdaf300bcf80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 073e4d0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 073e4d0e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 073e4d0e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3981 (Fri Oct 19 09:40:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c734d458", "local_conf_timestamp": 3982, "host-ts": 3981}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3982 (Fri Oct 19 09:40:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3982 (Fri Oct 19 09:40:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e40913da", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b0c61fbc-9b93-48f4-a0c8-992fb51648f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b0c61fbc-9b93-48f4-a0c8-992fb51648f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0ac1b100 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0ac1b100 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0ac1b100 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3993 (Fri Oct 19 09:40:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7a8514cf", "local_conf_timestamp": 3993, "host-ts": 3992}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5b18421a-9f4b-49ae-a5ac-ee53aa378171:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5b18421a-9f4b-49ae-a5ac-ee53aa378171:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0dbcd15a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0dbcd15a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0dbcd15a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3993 (Fri Oct 19 09:40:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7a8514cf", "local_conf_timestamp": 3993, "host-ts": 3992}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e87ef80e-c58d-43b8-8f9f-f7e74dbb73aa:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e87ef80e-c58d-43b8-8f9f-f7e74dbb73aa:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 10bd01d6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 10bd01d6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 10bd01d6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "143ef5d0", "local_conf_timestamp": 4002, "host-ts": 4002}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:72312e92-c5c7-420a-82e3-aacb317304d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:72312e92-c5c7-420a-82e3-aacb317304d6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 13f2bfee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 13f2bfee on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 13f2bfee on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "143ef5d0", "local_conf_timestamp": 4002, "host-ts": 4002}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Fri Oct 19 09:40:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3992 (Fri Oct 19 09:40:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e04fecdf", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:098baf60-62cf-4325-82da-fe6080f23166:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:098baf60-62cf-4325-82da-fe6080f23166:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 170833f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 170833f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 170833f8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4012 (Fri Oct 19 09:40:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4012 (Fri Oct 19 09:40:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba0dcb12", "local_conf_timestamp": 4012, "host-ts": 4012}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e9fd9e6b", "local_conf_timestamp": 4002, "host-ts": 4002}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:385b4460-1bb6-4b3a-a4ae-108491ed06a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:385b4460-1bb6-4b3a-a4ae-108491ed06a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1a2d1dc8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1a2d1dc8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1a2d1dc8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4012 (Fri Oct 19 09:40:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4012 (Fri Oct 19 09:40:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba0dcb12", "local_conf_timestamp": 4012, "host-ts": 4012}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4002 (Fri Oct 19 09:40:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4002 (Fri Oct 19 09:40:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e9fd9e6b", "local_conf_timestamp": 4002, "host-ts": 4002}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9799da1c-ab86-4b99-aaa8-aed6e378cf71:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9799da1c-ab86-4b99-aaa8-aed6e378cf71:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1d12f4ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1d12f4ea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1d12f4ea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4022 (Fri Oct 19 09:41:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4023 (Fri Oct 19 09:41:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e42ebe83", "local_conf_timestamp": 4023, "host-ts": 4022}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4022 (Fri Oct 19 09:41:08 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4022 (Fri Oct 19 09:41:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5254b8ef", "local_conf_timestamp": 4022, "host-ts": 4022}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1e393081-d77e-44e6-9a11-393ef1c2681e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1e393081-d77e-44e6-9a11-393ef1c2681e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 208f99e8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 208f99e8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 208f99e8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4022 (Fri Oct 19 09:41:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4023 (Fri Oct 19 09:41:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e42ebe83", "local_conf_timestamp": 4023, "host-ts": 4022}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4022 (Fri Oct 19 09:41:08 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4022 (Fri Oct 19 09:41:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "5254b8ef", "local_conf_timestamp": 4022, "host-ts": 4022}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fe84747a-4b28-44c3-91be-1cc097fa874e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fe84747a-4b28-44c3-91be-1cc097fa874e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2397ee42 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2397ee42 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2397ee42 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4033 (Fri Oct 19 09:41:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4033 (Fri Oct 19 09:41:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9bfb7d93", "local_conf_timestamp": 4033, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4032 (Fri Oct 19 09:41:19 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4032 (Fri Oct 19 09:41:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "561247ea", "local_conf_timestamp": 4032, "host-ts": 4032}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:33fa4360-4885-4c5b-ae5e-7489c048b04b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:33fa4360-4885-4c5b-ae5e-7489c048b04b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 24b53866 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 24b53866 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 24b53866 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4033 (Fri Oct 19 09:41:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4033 (Fri Oct 19 09:41:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9bfb7d93", "local_conf_timestamp": 4033, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4032 (Fri Oct 19 09:41:19 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4032 (Fri Oct 19 09:41:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "561247ea", "local_conf_timestamp": 4032, "host-ts": 4032}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:44aae76a-d180-4046-9599-f546858a588a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:44aae76a-d180-4046-9599-f546858a588a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2bcab28e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2bcab28e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2bcab28e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4043 (Fri Oct 19 09:41:30 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4043 (Fri Oct 19 09:41:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "f51c8fbd", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4042 (Fri Oct 19 09:41:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4042 (Fri Oct 19 09:41:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f91ead40", "local_conf_timestamp": 4042, "host-ts": 4042}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/416/ Build Number: 416 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/417/ Build Number: 417 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- All tests passed
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/418/ Build Number: 418 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... root: INFO: * Performing Activation... root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 79, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\nroot: INFO: * Performing Activation...\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/419/ Build Number: 419 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... root: INFO: * Performing Activation... root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 79, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\nroot: INFO: * Performing Activation...\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/420/ Build Number: 420 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/421/ Build Number: 421 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/422/ Build Number: 422 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:d52121ec-3b78-49a9-92be-b2e7527de084:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:d52121ec-3b78-49a9-92be-b2e7527de084:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running a0231f64 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a0231f64 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command a0231f64 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Sun Oct 21 17:42:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3965 (Sun Oct 21 17:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd96bd7b", "local_conf_timestamp": 3965, "host-ts": 3965}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3966 (Sun Oct 21 17:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3966 (Sun Oct 21 17:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false} lago.ssh: DEBUG: start task:08420dc3-7fac-4899-8967-6fb36a824ed1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:08420dc3-7fac-4899-8967-6fb36a824ed1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a13fb0b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command a13fb0b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:a7b1da04-9e15-4a0a-a004-4850b57ae468:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a7b1da04-9e15-4a0a-a004-4850b57ae468:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a28d0a12 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a28d0a12 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a28d0a12 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Sun Oct 21 17:42:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3965 (Sun Oct 21 17:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd96bd7b", "local_conf_timestamp": 3965, "host-ts": 3965}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3966 (Sun Oct 21 17:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3966 (Sun Oct 21 17:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false} lago.ssh: DEBUG: start task:9bbab095-a4a0-4149-8107-d959ad36535a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9bbab095-a4a0-4149-8107-d959ad36535a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a9e46346 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a9e46346 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a9e46346 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Sun Oct 21 17:42:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3975 (Sun Oct 21 17:42:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "029eaa68", "local_conf_timestamp": 3975, "host-ts": 3975}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3966 (Sun Oct 21 17:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3966 (Sun Oct 21 17:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false} lago.ssh: DEBUG: start task:b2b16a7f-a138-4dfd-8bbc-fd4c9cd8b78a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b2b16a7f-a138-4dfd-8bbc-fd4c9cd8b78a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ad8d660a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ad8d660a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ad8d660a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Sun Oct 21 17:42:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3985 (Sun Oct 21 17:42:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "618daac0", "local_conf_timestamp": 3985, "host-ts": 3985}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Sun Oct 21 17:42:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Sun Oct 21 17:42:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:d25218bd-b1b5-4478-84a7-b06c17abf737:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d25218bd-b1b5-4478-84a7-b06c17abf737:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b088ab30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b088ab30 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b088ab30 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Sun Oct 21 17:42:25 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3985 (Sun Oct 21 17:42:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "618daac0", "local_conf_timestamp": 3985, "host-ts": 3985}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Sun Oct 21 17:42:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Sun Oct 21 17:42:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:601ecc74-0aa2-481a-bc83-874dbef881e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:601ecc74-0aa2-481a-bc83-874dbef881e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b3c41aa0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b3c41aa0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b3c41aa0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3995 (Sun Oct 21 17:42:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3995 (Sun Oct 21 17:42:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cfbe9402", "local_conf_timestamp": 3995, "host-ts": 3995}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3986 (Sun Oct 21 17:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3986 (Sun Oct 21 17:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fefe5b01", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false} lago.ssh: DEBUG: start task:494d9075-ed70-4f4f-89d4-4cbdca163f11:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:494d9075-ed70-4f4f-89d4-4cbdca163f11:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b6d3574c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b6d3574c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b6d3574c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3995 (Sun Oct 21 17:42:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3995 (Sun Oct 21 17:42:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cfbe9402", "local_conf_timestamp": 3995, "host-ts": 3995}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3986 (Sun Oct 21 17:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3986 (Sun Oct 21 17:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fefe5b01", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false} lago.ssh: DEBUG: start task:e5d6a222-3070-415d-b080-efcfd756c47e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e5d6a222-3070-415d-b080-efcfd756c47e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ba049f20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ba049f20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ba049f20 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4005 (Sun Oct 21 17:42:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4005 (Sun Oct 21 17:42:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d602458b", "local_conf_timestamp": 4005, "host-ts": 4005}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Sun Oct 21 17:42:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3997 (Sun Oct 21 17:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:4a7fcfb9-ca6e-4010-bd1e-4d6f4822ed75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4a7fcfb9-ca6e-4010-bd1e-4d6f4822ed75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bcfac3a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bcfac3a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bcfac3a8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4005 (Sun Oct 21 17:42:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4005 (Sun Oct 21 17:42:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d602458b", "local_conf_timestamp": 4005, "host-ts": 4005}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Sun Oct 21 17:42:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3997 (Sun Oct 21 17:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:1472cbf9-d7fc-4ff9-b6a5-467a86d8966e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1472cbf9-d7fc-4ff9-b6a5-467a86d8966e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c0243a5a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c0243a5a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c0243a5a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4015 (Sun Oct 21 17:42:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4015 (Sun Oct 21 17:42:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78317b49", "local_conf_timestamp": 4015, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4007 (Sun Oct 21 17:42:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4007 (Sun Oct 21 17:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ed3c636", "local_conf_timestamp": 4007, "host-ts": 4007}, "global_maintenance": false} lago.ssh: DEBUG: start task:ca9c2000-d6d8-4051-98c9-2f7c4823ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ca9c2000-d6d8-4051-98c9-2f7c4823ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c309eb66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c309eb66 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c309eb66 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4015 (Sun Oct 21 17:42:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4015 (Sun Oct 21 17:42:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78317b49", "local_conf_timestamp": 4015, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4007 (Sun Oct 21 17:42:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4007 (Sun Oct 21 17:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ed3c636", "local_conf_timestamp": 4007, "host-ts": 4007}, "global_maintenance": false} lago.ssh: DEBUG: start task:9d4e173c-4f5e-4293-9778-019592272ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9d4e173c-4f5e-4293-9778-019592272ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c5f9d732 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c5f9d732 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c5f9d732 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4025 (Sun Oct 21 17:43:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4026 (Sun Oct 21 17:43:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4017 (Sun Oct 21 17:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4017 (Sun Oct 21 17:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9b1e1bd", "local_conf_timestamp": 4017, "host-ts": 4017}, "global_maintenance": false} lago.ssh: DEBUG: start task:0e0b2df4-1953-407e-b571-1048abb34040:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0e0b2df4-1953-407e-b571-1048abb34040:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c8fd1660 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c8fd1660 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c8fd1660 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4025 (Sun Oct 21 17:43:05 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4026 (Sun Oct 21 17:43:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4017 (Sun Oct 21 17:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4017 (Sun Oct 21 17:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9b1e1bd", "local_conf_timestamp": 4017, "host-ts": 4017}, "global_maintenance": false} lago.ssh: DEBUG: start task:86a7e55f-b6dd-4558-82fe-1f7355f62a8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:86a7e55f-b6dd-4558-82fe-1f7355f62a8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cc3ab2a6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cc3ab2a6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cc3ab2a6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Sun Oct 21 17:43:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4036 (Sun Oct 21 17:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4027 (Sun Oct 21 17:43:07 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4027 (Sun Oct 21 17:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d57ae0b2", "local_conf_timestamp": 4027, "host-ts": 4027}, "global_maintenance": false} lago.ssh: DEBUG: start task:26b7bbf0-d061-4d4d-952e-381eb8458f60:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:26b7bbf0-d061-4d4d-952e-381eb8458f60:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cf176190 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cf176190 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cf176190 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Sun Oct 21 17:43:15 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4036 (Sun Oct 21 17:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4027 (Sun Oct 21 17:43:07 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4027 (Sun Oct 21 17:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d57ae0b2", "local_conf_timestamp": 4027, "host-ts": 4027}, "global_maintenance": false} lago.ssh: DEBUG: start task:ca20cf91-3c89-4e7b-9427-52626c030958:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ca20cf91-3c89-4e7b-9427-52626c030958:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d1f59bde on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d1f59bde on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d1f59bde on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Sun Oct 21 17:43:26 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4046 (Sun Oct 21 17:43:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4037 (Sun Oct 21 17:43:17 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4038 (Sun Oct 21 17:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41830226", "local_conf_timestamp": 4038, "host-ts": 4037}, "global_maintenance": false} lago.ssh: DEBUG: start task:bc816e74-460a-4a01-9b3e-0014509668fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bc816e74-460a-4a01-9b3e-0014509668fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d301c458 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d301c458 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d301c458 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Sun Oct 21 17:43:26 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4046 (Sun Oct 21 17:43:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4037 (Sun Oct 21 17:43:17 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4038 (Sun Oct 21 17:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41830226", "local_conf_timestamp": 4038, "host-ts": 4037}, "global_maintenance": false} lago.ssh: DEBUG: start task:b991c3d3-4880-4847-9205-d83eb65f6409:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b991c3d3-4880-4847-9205-d83eb65f6409:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running da075880 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command da075880 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command da075880 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4056 (Sun Oct 21 17:43:35 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4056 (Sun Oct 21 17:43:35 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "9db5660a", "local_conf_timestamp": 4056, "host-ts": 4056}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4047 (Sun Oct 21 17:43:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4047 (Sun Oct 21 17:43:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e0f79715", "local_conf_timestamp": 4047, "host-ts": 4047}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:d52121ec-3b78-49a9-92be-b2e7527de084:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:d52121ec-3b78-49a9-92be-b2e7527de084:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running a0231f64 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a0231f64 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command a0231f64 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Sun Oct 21 17:42:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3965 (Sun Oct 21 17:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd96bd7b", "local_conf_timestamp": 3965, "host-ts": 3965}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3966 (Sun Oct 21 17:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3966 (Sun Oct 21 17:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:08420dc3-7fac-4899-8967-6fb36a824ed1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:08420dc3-7fac-4899-8967-6fb36a824ed1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a13fb0b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command a13fb0b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:a7b1da04-9e15-4a0a-a004-4850b57ae468:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a7b1da04-9e15-4a0a-a004-4850b57ae468:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a28d0a12 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a28d0a12 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a28d0a12 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Sun Oct 21 17:42:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3965 (Sun Oct 21 17:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fd96bd7b", "local_conf_timestamp": 3965, "host-ts": 3965}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3966 (Sun Oct 21 17:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3966 (Sun Oct 21 17:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9bbab095-a4a0-4149-8107-d959ad36535a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9bbab095-a4a0-4149-8107-d959ad36535a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a9e46346 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a9e46346 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a9e46346 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Sun Oct 21 17:42:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3975 (Sun Oct 21 17:42:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "029eaa68", "local_conf_timestamp": 3975, "host-ts": 3975}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3966 (Sun Oct 21 17:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3966 (Sun Oct 21 17:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b2b16a7f-a138-4dfd-8bbc-fd4c9cd8b78a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b2b16a7f-a138-4dfd-8bbc-fd4c9cd8b78a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ad8d660a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ad8d660a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ad8d660a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Sun Oct 21 17:42:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3985 (Sun Oct 21 17:42:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "618daac0", "local_conf_timestamp": 3985, "host-ts": 3985}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Sun Oct 21 17:42:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Sun Oct 21 17:42:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d25218bd-b1b5-4478-84a7-b06c17abf737:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d25218bd-b1b5-4478-84a7-b06c17abf737:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b088ab30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b088ab30 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b088ab30 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Sun Oct 21 17:42:25 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3985 (Sun Oct 21 17:42:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "618daac0", "local_conf_timestamp": 3985, "host-ts": 3985}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Sun Oct 21 17:42:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Sun Oct 21 17:42:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:601ecc74-0aa2-481a-bc83-874dbef881e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:601ecc74-0aa2-481a-bc83-874dbef881e7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b3c41aa0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b3c41aa0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b3c41aa0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3995 (Sun Oct 21 17:42:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3995 (Sun Oct 21 17:42:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cfbe9402", "local_conf_timestamp": 3995, "host-ts": 3995}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3986 (Sun Oct 21 17:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3986 (Sun Oct 21 17:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fefe5b01", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:494d9075-ed70-4f4f-89d4-4cbdca163f11:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:494d9075-ed70-4f4f-89d4-4cbdca163f11:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b6d3574c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b6d3574c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b6d3574c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3995 (Sun Oct 21 17:42:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3995 (Sun Oct 21 17:42:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cfbe9402", "local_conf_timestamp": 3995, "host-ts": 3995}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3986 (Sun Oct 21 17:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3986 (Sun Oct 21 17:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fefe5b01", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e5d6a222-3070-415d-b080-efcfd756c47e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e5d6a222-3070-415d-b080-efcfd756c47e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ba049f20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ba049f20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ba049f20 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4005 (Sun Oct 21 17:42:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4005 (Sun Oct 21 17:42:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d602458b", "local_conf_timestamp": 4005, "host-ts": 4005}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Sun Oct 21 17:42:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3997 (Sun Oct 21 17:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4a7fcfb9-ca6e-4010-bd1e-4d6f4822ed75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4a7fcfb9-ca6e-4010-bd1e-4d6f4822ed75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bcfac3a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bcfac3a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bcfac3a8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4005 (Sun Oct 21 17:42:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4005 (Sun Oct 21 17:42:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d602458b", "local_conf_timestamp": 4005, "host-ts": 4005}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Sun Oct 21 17:42:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3997 (Sun Oct 21 17:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1472cbf9-d7fc-4ff9-b6a5-467a86d8966e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1472cbf9-d7fc-4ff9-b6a5-467a86d8966e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c0243a5a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c0243a5a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c0243a5a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4015 (Sun Oct 21 17:42:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4015 (Sun Oct 21 17:42:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78317b49", "local_conf_timestamp": 4015, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4007 (Sun Oct 21 17:42:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4007 (Sun Oct 21 17:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ed3c636", "local_conf_timestamp": 4007, "host-ts": 4007}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ca9c2000-d6d8-4051-98c9-2f7c4823ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ca9c2000-d6d8-4051-98c9-2f7c4823ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c309eb66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c309eb66 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c309eb66 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4015 (Sun Oct 21 17:42:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4015 (Sun Oct 21 17:42:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78317b49", "local_conf_timestamp": 4015, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4007 (Sun Oct 21 17:42:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4007 (Sun Oct 21 17:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ed3c636", "local_conf_timestamp": 4007, "host-ts": 4007}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9d4e173c-4f5e-4293-9778-019592272ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9d4e173c-4f5e-4293-9778-019592272ce7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c5f9d732 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c5f9d732 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c5f9d732 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4025 (Sun Oct 21 17:43:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4026 (Sun Oct 21 17:43:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4017 (Sun Oct 21 17:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4017 (Sun Oct 21 17:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9b1e1bd", "local_conf_timestamp": 4017, "host-ts": 4017}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0e0b2df4-1953-407e-b571-1048abb34040:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0e0b2df4-1953-407e-b571-1048abb34040:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c8fd1660 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c8fd1660 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c8fd1660 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4025 (Sun Oct 21 17:43:05 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4026 (Sun Oct 21 17:43:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4017 (Sun Oct 21 17:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4017 (Sun Oct 21 17:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9b1e1bd", "local_conf_timestamp": 4017, "host-ts": 4017}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:86a7e55f-b6dd-4558-82fe-1f7355f62a8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:86a7e55f-b6dd-4558-82fe-1f7355f62a8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cc3ab2a6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cc3ab2a6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cc3ab2a6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Sun Oct 21 17:43:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4036 (Sun Oct 21 17:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4027 (Sun Oct 21 17:43:07 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4027 (Sun Oct 21 17:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d57ae0b2", "local_conf_timestamp": 4027, "host-ts": 4027}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:26b7bbf0-d061-4d4d-952e-381eb8458f60:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:26b7bbf0-d061-4d4d-952e-381eb8458f60:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cf176190 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cf176190 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cf176190 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Sun Oct 21 17:43:15 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4036 (Sun Oct 21 17:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4027 (Sun Oct 21 17:43:07 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4027 (Sun Oct 21 17:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d57ae0b2", "local_conf_timestamp": 4027, "host-ts": 4027}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ca20cf91-3c89-4e7b-9427-52626c030958:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ca20cf91-3c89-4e7b-9427-52626c030958:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d1f59bde on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d1f59bde on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d1f59bde on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Sun Oct 21 17:43:26 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4046 (Sun Oct 21 17:43:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4037 (Sun Oct 21 17:43:17 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4038 (Sun Oct 21 17:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41830226", "local_conf_timestamp": 4038, "host-ts": 4037}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bc816e74-460a-4a01-9b3e-0014509668fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bc816e74-460a-4a01-9b3e-0014509668fc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d301c458 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d301c458 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d301c458 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Sun Oct 21 17:43:26 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4046 (Sun Oct 21 17:43:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4037 (Sun Oct 21 17:43:17 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4038 (Sun Oct 21 17:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41830226", "local_conf_timestamp": 4038, "host-ts": 4037}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b991c3d3-4880-4847-9205-d83eb65f6409:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b991c3d3-4880-4847-9205-d83eb65f6409:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running da075880 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command da075880 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command da075880 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4056 (Sun Oct 21 17:43:35 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4056 (Sun Oct 21 17:43:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "9db5660a", "local_conf_timestamp": 4056, "host-ts": 4056}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4047 (Sun Oct 21 17:43:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4047 (Sun Oct 21 17:43:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e0f79715", "local_conf_timestamp": 4047, "host-ts": 4047}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/423/ Build Number: 423 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:f5e6ed06-7ce6-434b-8c4c-a75842eee407:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:f5e6ed06-7ce6-434b-8c4c-a75842eee407:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 93758390 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 93758390 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 93758390 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3962 (Mon Oct 22 01:41:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3962 (Mon Oct 22 01:41:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5d956133", "local_conf_timestamp": 3962, "host-ts": 3962}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3959 (Mon Oct 22 01:41:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3959 (Mon Oct 22 01:41:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecaa7b06", "local_conf_timestamp": 3959, "host-ts": 3959}, "global_maintenance": false} lago.ssh: DEBUG: start task:35b5da50-f6ae-4773-8b89-39596a59f402:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:35b5da50-f6ae-4773-8b89-39596a59f402:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 947d9552 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 947d9552 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:bb4ef247-71d3-4826-8f76-afd4ed958ee1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bb4ef247-71d3-4826-8f76-afd4ed958ee1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 958a82ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 958a82ca on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 958a82ca on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Mon Oct 22 01:41:24 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=3973 (Mon Oct 22 01:41:25 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenanceMigrateVm\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bdb7d5a3", "local_conf_timestamp": 3973, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3969 (Mon Oct 22 01:41:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3970 (Mon Oct 22 01:41:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "80a6f3ec", "local_conf_timestamp": 3970, "host-ts": 3969}, "global_maintenance": false} lago.ssh: DEBUG: start task:388c8051-3bd3-4b42-bbc1-936044981d46:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:388c8051-3bd3-4b42-bbc1-936044981d46:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9cb9a292 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9cb9a292 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9cb9a292 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Mon Oct 22 01:41:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Mon Oct 22 01:41:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3979 (Mon Oct 22 01:41:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3979 (Mon Oct 22 01:41:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "931ab094", "local_conf_timestamp": 3979, "host-ts": 3979}, "global_maintenance": false} lago.ssh: DEBUG: start task:7e46f88c-b883-41b1-9972-ba650a289477:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7e46f88c-b883-41b1-9972-ba650a289477:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a0029ca6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a0029ca6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a0029ca6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Mon Oct 22 01:41:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Mon Oct 22 01:41:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3979 (Mon Oct 22 01:41:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3979 (Mon Oct 22 01:41:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "931ab094", "local_conf_timestamp": 3979, "host-ts": 3979}, "global_maintenance": false} lago.ssh: DEBUG: start task:349f4fdf-4cad-4bf4-83fb-c2fb940c44b8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:349f4fdf-4cad-4bf4-83fb-c2fb940c44b8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a2fbb0b4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a2fbb0b4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a2fbb0b4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3994 (Mon Oct 22 01:41:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Mon Oct 22 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3989 (Mon Oct 22 01:41:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3989 (Mon Oct 22 01:41:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "acfdb5a7", "local_conf_timestamp": 3989, "host-ts": 3989}, "global_maintenance": false} lago.ssh: DEBUG: start task:f71e53e1-0318-48d4-8f21-9d1e9328d674:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f71e53e1-0318-48d4-8f21-9d1e9328d674:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a5fd720c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a5fd720c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a5fd720c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3994 (Mon Oct 22 01:41:45 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Mon Oct 22 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3989 (Mon Oct 22 01:41:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3989 (Mon Oct 22 01:41:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "acfdb5a7", "local_conf_timestamp": 3989, "host-ts": 3989}, "global_maintenance": false} lago.ssh: DEBUG: start task:4a6cb8a6-b2c2-4991-bf84-8ecaaa23e5ab:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4a6cb8a6-b2c2-4991-bf84-8ecaaa23e5ab:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aa0f8a2e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aa0f8a2e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command aa0f8a2e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4004 (Mon Oct 22 01:41:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4006 (Mon Oct 22 01:41:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "da58ac4f", "local_conf_timestamp": 4006, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3999 (Mon Oct 22 01:41:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3999 (Mon Oct 22 01:41:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a8bb4aa2", "local_conf_timestamp": 3999, "host-ts": 3999}, "global_maintenance": false} lago.ssh: DEBUG: start task:acefaf09-3ad7-4c65-a4a4-7f2825db6b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:acefaf09-3ad7-4c65-a4a4-7f2825db6b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running acf03b80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command acf03b80 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command acf03b80 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4004 (Mon Oct 22 01:41:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4006 (Mon Oct 22 01:41:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "da58ac4f", "local_conf_timestamp": 4006, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3999 (Mon Oct 22 01:41:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3999 (Mon Oct 22 01:41:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a8bb4aa2", "local_conf_timestamp": 3999, "host-ts": 3999}, "global_maintenance": false} lago.ssh: DEBUG: start task:d2704a5c-7ebd-4162-ac28-d8708201cafb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d2704a5c-7ebd-4162-ac28-d8708201cafb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running afdc3948 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command afdc3948 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command afdc3948 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4015 (Mon Oct 22 01:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4016 (Mon Oct 22 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e1382af3", "local_conf_timestamp": 4016, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4009 (Mon Oct 22 01:42:01 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4010 (Mon Oct 22 01:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "72ea697d", "local_conf_timestamp": 4010, "host-ts": 4009}, "global_maintenance": false} lago.ssh: DEBUG: start task:b3a22a7b-0665-4b82-8f63-951fad8887ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b3a22a7b-0665-4b82-8f63-951fad8887ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b34bdb38 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b34bdb38 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b34bdb38 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4015 (Mon Oct 22 01:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4016 (Mon Oct 22 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e1382af3", "local_conf_timestamp": 4016, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4009 (Mon Oct 22 01:42:01 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4010 (Mon Oct 22 01:42:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "72ea697d", "local_conf_timestamp": 4010, "host-ts": 4009}, "global_maintenance": false} lago.ssh: DEBUG: start task:576f2194-dcbf-4ae1-b9c8-2b7cf31fb294:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:576f2194-dcbf-4ae1-b9c8-2b7cf31fb294:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b69b9652 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b69b9652 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b69b9652 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4025 (Mon Oct 22 01:42:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4026 (Mon Oct 22 01:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4019 (Mon Oct 22 01:42:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4019 (Mon Oct 22 01:42:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "166b1f9d", "local_conf_timestamp": 4019, "host-ts": 4019}, "global_maintenance": false} lago.ssh: DEBUG: start task:03e1bddc-bc1e-4fc6-8c09-43bcd631bf66:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:03e1bddc-bc1e-4fc6-8c09-43bcd631bf66:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b9a008a6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b9a008a6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b9a008a6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4025 (Mon Oct 22 01:42:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4026 (Mon Oct 22 01:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4019 (Mon Oct 22 01:42:10 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4019 (Mon Oct 22 01:42:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "166b1f9d", "local_conf_timestamp": 4019, "host-ts": 4019}, "global_maintenance": false} lago.ssh: DEBUG: start task:e7fed855-45ee-4d80-93f9-016bcdbcc7f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e7fed855-45ee-4d80-93f9-016bcdbcc7f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bd137338 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bd137338 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bd137338 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4035 (Mon Oct 22 01:42:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4035 (Mon Oct 22 01:42:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ff26008c", "local_conf_timestamp": 4035, "host-ts": 4035}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4029 (Mon Oct 22 01:42:20 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4029 (Mon Oct 22 01:42:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1aa01e92", "local_conf_timestamp": 4029, "host-ts": 4029}, "global_maintenance": false} lago.ssh: DEBUG: start task:a6686887-52db-4ec3-9d98-6ccc2fa3a74c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a6686887-52db-4ec3-9d98-6ccc2fa3a74c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bfe4970e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bfe4970e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bfe4970e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4035 (Mon Oct 22 01:42:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4035 (Mon Oct 22 01:42:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ff26008c", "local_conf_timestamp": 4035, "host-ts": 4035}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4029 (Mon Oct 22 01:42:20 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4029 (Mon Oct 22 01:42:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1aa01e92", "local_conf_timestamp": 4029, "host-ts": 4029}, "global_maintenance": false} lago.ssh: DEBUG: start task:169fe158-796e-4c03-9c1a-2424547b07b4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:169fe158-796e-4c03-9c1a-2424547b07b4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c2b82ca2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c2b82ca2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c2b82ca2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4045 (Mon Oct 22 01:42:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4046 (Mon Oct 22 01:42:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f08f12c3", "local_conf_timestamp": 4046, "host-ts": 4045}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4039 (Mon Oct 22 01:42:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4039 (Mon Oct 22 01:42:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0326e3ba", "local_conf_timestamp": 4039, "host-ts": 4039}, "global_maintenance": false} lago.ssh: DEBUG: start task:58b2aa32-e57e-43af-a2a5-d7ea93c779c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:58b2aa32-e57e-43af-a2a5-d7ea93c779c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c3bcc892 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c3bcc892 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c3bcc892 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4045 (Mon Oct 22 01:42:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4046 (Mon Oct 22 01:42:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f08f12c3", "local_conf_timestamp": 4046, "host-ts": 4045}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4039 (Mon Oct 22 01:42:31 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4039 (Mon Oct 22 01:42:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0326e3ba", "local_conf_timestamp": 4039, "host-ts": 4039}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:f5e6ed06-7ce6-434b-8c4c-a75842eee407:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:f5e6ed06-7ce6-434b-8c4c-a75842eee407:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 93758390 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 93758390 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 93758390 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3962 (Mon Oct 22 01:41:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3962 (Mon Oct 22 01:41:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5d956133", "local_conf_timestamp": 3962, "host-ts": 3962}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3959 (Mon Oct 22 01:41:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3959 (Mon Oct 22 01:41:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ecaa7b06", "local_conf_timestamp": 3959, "host-ts": 3959}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:35b5da50-f6ae-4773-8b89-39596a59f402:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:35b5da50-f6ae-4773-8b89-39596a59f402:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 947d9552 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 947d9552 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:bb4ef247-71d3-4826-8f76-afd4ed958ee1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bb4ef247-71d3-4826-8f76-afd4ed958ee1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 958a82ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 958a82ca on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 958a82ca on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Mon Oct 22 01:41:24 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=3973 (Mon Oct 22 01:41:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenanceMigrateVm\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "bdb7d5a3", "local_conf_timestamp": 3973, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3969 (Mon Oct 22 01:41:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3970 (Mon Oct 22 01:41:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "80a6f3ec", "local_conf_timestamp": 3970, "host-ts": 3969}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:388c8051-3bd3-4b42-bbc1-936044981d46:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:388c8051-3bd3-4b42-bbc1-936044981d46:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9cb9a292 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9cb9a292 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9cb9a292 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Mon Oct 22 01:41:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Mon Oct 22 01:41:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3979 (Mon Oct 22 01:41:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3979 (Mon Oct 22 01:41:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "931ab094", "local_conf_timestamp": 3979, "host-ts": 3979}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7e46f88c-b883-41b1-9972-ba650a289477:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7e46f88c-b883-41b1-9972-ba650a289477:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a0029ca6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a0029ca6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a0029ca6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Mon Oct 22 01:41:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Mon Oct 22 01:41:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3979 (Mon Oct 22 01:41:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3979 (Mon Oct 22 01:41:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "931ab094", "local_conf_timestamp": 3979, "host-ts": 3979}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:349f4fdf-4cad-4bf4-83fb-c2fb940c44b8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:349f4fdf-4cad-4bf4-83fb-c2fb940c44b8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a2fbb0b4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a2fbb0b4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a2fbb0b4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3994 (Mon Oct 22 01:41:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Mon Oct 22 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3989 (Mon Oct 22 01:41:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3989 (Mon Oct 22 01:41:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "acfdb5a7", "local_conf_timestamp": 3989, "host-ts": 3989}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f71e53e1-0318-48d4-8f21-9d1e9328d674:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f71e53e1-0318-48d4-8f21-9d1e9328d674:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a5fd720c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a5fd720c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a5fd720c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3994 (Mon Oct 22 01:41:45 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Mon Oct 22 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2dea1cea", "local_conf_timestamp": 3994, "host-ts": 3994}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3989 (Mon Oct 22 01:41:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3989 (Mon Oct 22 01:41:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "acfdb5a7", "local_conf_timestamp": 3989, "host-ts": 3989}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4a6cb8a6-b2c2-4991-bf84-8ecaaa23e5ab:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4a6cb8a6-b2c2-4991-bf84-8ecaaa23e5ab:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aa0f8a2e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aa0f8a2e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command aa0f8a2e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4004 (Mon Oct 22 01:41:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4006 (Mon Oct 22 01:41:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "da58ac4f", "local_conf_timestamp": 4006, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3999 (Mon Oct 22 01:41:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3999 (Mon Oct 22 01:41:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a8bb4aa2", "local_conf_timestamp": 3999, "host-ts": 3999}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:acefaf09-3ad7-4c65-a4a4-7f2825db6b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:acefaf09-3ad7-4c65-a4a4-7f2825db6b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running acf03b80 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command acf03b80 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command acf03b80 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4004 (Mon Oct 22 01:41:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4006 (Mon Oct 22 01:41:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "da58ac4f", "local_conf_timestamp": 4006, "host-ts": 4004}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3999 (Mon Oct 22 01:41:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3999 (Mon Oct 22 01:41:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a8bb4aa2", "local_conf_timestamp": 3999, "host-ts": 3999}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d2704a5c-7ebd-4162-ac28-d8708201cafb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d2704a5c-7ebd-4162-ac28-d8708201cafb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running afdc3948 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command afdc3948 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command afdc3948 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4015 (Mon Oct 22 01:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4016 (Mon Oct 22 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e1382af3", "local_conf_timestamp": 4016, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4009 (Mon Oct 22 01:42:01 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4010 (Mon Oct 22 01:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "72ea697d", "local_conf_timestamp": 4010, "host-ts": 4009}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b3a22a7b-0665-4b82-8f63-951fad8887ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b3a22a7b-0665-4b82-8f63-951fad8887ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b34bdb38 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b34bdb38 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b34bdb38 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4015 (Mon Oct 22 01:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4016 (Mon Oct 22 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e1382af3", "local_conf_timestamp": 4016, "host-ts": 4015}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4009 (Mon Oct 22 01:42:01 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4010 (Mon Oct 22 01:42:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "72ea697d", "local_conf_timestamp": 4010, "host-ts": 4009}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:576f2194-dcbf-4ae1-b9c8-2b7cf31fb294:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:576f2194-dcbf-4ae1-b9c8-2b7cf31fb294:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b69b9652 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b69b9652 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b69b9652 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4025 (Mon Oct 22 01:42:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4026 (Mon Oct 22 01:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4019 (Mon Oct 22 01:42:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4019 (Mon Oct 22 01:42:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "166b1f9d", "local_conf_timestamp": 4019, "host-ts": 4019}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:03e1bddc-bc1e-4fc6-8c09-43bcd631bf66:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:03e1bddc-bc1e-4fc6-8c09-43bcd631bf66:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b9a008a6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b9a008a6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b9a008a6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4025 (Mon Oct 22 01:42:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4026 (Mon Oct 22 01:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c81c6ff4", "local_conf_timestamp": 4026, "host-ts": 4025}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4019 (Mon Oct 22 01:42:10 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4019 (Mon Oct 22 01:42:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "166b1f9d", "local_conf_timestamp": 4019, "host-ts": 4019}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e7fed855-45ee-4d80-93f9-016bcdbcc7f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e7fed855-45ee-4d80-93f9-016bcdbcc7f6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bd137338 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bd137338 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bd137338 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4035 (Mon Oct 22 01:42:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4035 (Mon Oct 22 01:42:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ff26008c", "local_conf_timestamp": 4035, "host-ts": 4035}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4029 (Mon Oct 22 01:42:20 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4029 (Mon Oct 22 01:42:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1aa01e92", "local_conf_timestamp": 4029, "host-ts": 4029}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a6686887-52db-4ec3-9d98-6ccc2fa3a74c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a6686887-52db-4ec3-9d98-6ccc2fa3a74c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bfe4970e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bfe4970e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bfe4970e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4035 (Mon Oct 22 01:42:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4035 (Mon Oct 22 01:42:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ff26008c", "local_conf_timestamp": 4035, "host-ts": 4035}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4029 (Mon Oct 22 01:42:20 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4029 (Mon Oct 22 01:42:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1aa01e92", "local_conf_timestamp": 4029, "host-ts": 4029}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:169fe158-796e-4c03-9c1a-2424547b07b4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:169fe158-796e-4c03-9c1a-2424547b07b4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c2b82ca2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c2b82ca2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c2b82ca2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4045 (Mon Oct 22 01:42:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4046 (Mon Oct 22 01:42:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f08f12c3", "local_conf_timestamp": 4046, "host-ts": 4045}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4039 (Mon Oct 22 01:42:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4039 (Mon Oct 22 01:42:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0326e3ba", "local_conf_timestamp": 4039, "host-ts": 4039}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:58b2aa32-e57e-43af-a2a5-d7ea93c779c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:58b2aa32-e57e-43af-a2a5-d7ea93c779c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c3bcc892 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c3bcc892 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c3bcc892 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4045 (Mon Oct 22 01:42:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4046 (Mon Oct 22 01:42:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f08f12c3", "local_conf_timestamp": 4046, "host-ts": 4045}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4039 (Mon Oct 22 01:42:31 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4039 (Mon Oct 22 01:42:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0326e3ba", "local_conf_timestamp": 4039, "host-ts": 4039}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/424/ Build Number: 424 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #424 [Galit] Fix stdci.yaml to be able to run poll upstream sources ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:0966d06f-3a9e-4a45-8fec-8b2fc2e860df:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:0966d06f-3a9e-4a45-8fec-8b2fc2e860df:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 747e263e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 747e263e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 747e263e on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3956 (Mon Oct 22 09:39:54 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3957 (Mon Oct 22 09:39:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3dee5dde", "local_conf_timestamp": 3957, "host-ts": 3956}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3966 (Mon Oct 22 09:40:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3966 (Mon Oct 22 09:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false} lago.ssh: DEBUG: start task:90226002-1343-4f2a-96ac-672422f573d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:90226002-1343-4f2a-96ac-672422f573d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 758ea1ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 758ea1ca on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:44b84adf-ab5e-4e71-b5be-38c731e9f409:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:44b84adf-ab5e-4e71-b5be-38c731e9f409:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 76bc2388 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 76bc2388 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 76bc2388 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3966 (Mon Oct 22 09:40:04 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3966 (Mon Oct 22 09:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f531ab48", "local_conf_timestamp": 3966, "host-ts": 3966}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3966 (Mon Oct 22 09:40:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3966 (Mon Oct 22 09:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false} lago.ssh: DEBUG: start task:ef946b3c-ef4b-4192-83b5-8e22d763cb3a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ef946b3c-ef4b-4192-83b5-8e22d763cb3a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7e228d4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7e228d4c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7e228d4c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0a39bc5b", "local_conf_timestamp": 3976, "host-ts": 3976}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:1d1ff876-e5dc-44ce-8cb1-a1d3b0dc8fca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1d1ff876-e5dc-44ce-8cb1-a1d3b0dc8fca:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8158654a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8158654a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8158654a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3986 (Mon Oct 22 09:40:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3986 (Mon Oct 22 09:40:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9c0135b9", "local_conf_timestamp": 3986, "host-ts": 3986}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:b615ebf6-3c87-42e5-b41a-abba1e9300de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b615ebf6-3c87-42e5-b41a-abba1e9300de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 84513862 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 84513862 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 84513862 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3986 (Mon Oct 22 09:40:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3986 (Mon Oct 22 09:40:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9c0135b9", "local_conf_timestamp": 3986, "host-ts": 3986}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:5824d6b4-3613-4f6f-8bc8-cf546e7835d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5824d6b4-3613-4f6f-8bc8-cf546e7835d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 876b4736 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 876b4736 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 876b4736 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Mon Oct 22 09:40:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32320b7b", "local_conf_timestamp": 3996, "host-ts": 3996}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Mon Oct 22 09:40:33 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fab8a404", "local_conf_timestamp": 3996, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:4d532bef-d9eb-4be6-809b-92f139b34f29:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4d532bef-d9eb-4be6-809b-92f139b34f29:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a77a6f4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8a77a6f4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8a77a6f4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Mon Oct 22 09:40:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32320b7b", "local_conf_timestamp": 3996, "host-ts": 3996}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Mon Oct 22 09:40:33 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fab8a404", "local_conf_timestamp": 3996, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:78d28bd8-9299-494b-9a81-b4ebb72b027a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:78d28bd8-9299-494b-9a81-b4ebb72b027a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8da5da12 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8da5da12 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8da5da12 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b8edaf2", "local_conf_timestamp": 4006, "host-ts": 4006}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} lago.ssh: DEBUG: start task:3b969c76-112a-4f6f-b19d-a0a7e8d68880:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3b969c76-112a-4f6f-b19d-a0a7e8d68880:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 90ef0c02 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 90ef0c02 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 90ef0c02 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b8edaf2", "local_conf_timestamp": 4006, "host-ts": 4006}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} lago.ssh: DEBUG: start task:25e9ad5a-579e-4679-ba63-f8643bd3fc6c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:25e9ad5a-579e-4679-ba63-f8643bd3fc6c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 93f402ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 93f402ea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 93f402ea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4017 (Mon Oct 22 09:40:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2bad4a6", "local_conf_timestamp": 4017, "host-ts": 4016}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4016 (Mon Oct 22 09:40:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4468f13b", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:13ecc994-5878-48d3-8fda-421414522abd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:13ecc994-5878-48d3-8fda-421414522abd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 974b83d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 974b83d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 974b83d2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4026 (Mon Oct 22 09:41:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4026 (Mon Oct 22 09:41:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac99a137", "local_conf_timestamp": 4026, "host-ts": 4026}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4016 (Mon Oct 22 09:40:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4468f13b", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:8d7a9dd6-fd6b-4150-84ca-fd153ed758c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8d7a9dd6-fd6b-4150-84ca-fd153ed758c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9b03dcea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9b03dcea on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9b03dcea on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4026 (Mon Oct 22 09:41:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4026 (Mon Oct 22 09:41:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac99a137", "local_conf_timestamp": 4026, "host-ts": 4026}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4016 (Mon Oct 22 09:40:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4468f13b", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:2a402aa9-c8da-460c-96f7-432fbaa0bb00:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2a402aa9-c8da-460c-96f7-432fbaa0bb00:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9e86bee6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9e86bee6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9e86bee6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Mon Oct 22 09:41:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Mon Oct 22 09:41:13 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4ce50f31", "local_conf_timestamp": 4036, "host-ts": 4036}, "global_maintenance": false} lago.ssh: DEBUG: start task:bf2431c7-bfac-4d73-b109-abfcc141f0e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bf2431c7-bfac-4d73-b109-abfcc141f0e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a15f7c3e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a15f7c3e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a15f7c3e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Mon Oct 22 09:41:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Mon Oct 22 09:41:13 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4ce50f31", "local_conf_timestamp": 4036, "host-ts": 4036}, "global_maintenance": false} lago.ssh: DEBUG: start task:82edd5bb-2ebf-40c6-9c4c-74d07254be34:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:82edd5bb-2ebf-40c6-9c4c-74d07254be34:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a45b661e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a45b661e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a45b661e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false} lago.ssh: DEBUG: start task:1369e16e-1a67-4f9b-bb7e-0388c32e41c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1369e16e-1a67-4f9b-bb7e-0388c32e41c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a572a774 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a572a774 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a572a774 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:0966d06f-3a9e-4a45-8fec-8b2fc2e860df:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:0966d06f-3a9e-4a45-8fec-8b2fc2e860df:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 747e263e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 747e263e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 747e263e on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3956 (Mon Oct 22 09:39:54 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3957 (Mon Oct 22 09:39:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3dee5dde", "local_conf_timestamp": 3957, "host-ts": 3956}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3966 (Mon Oct 22 09:40:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3966 (Mon Oct 22 09:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:90226002-1343-4f2a-96ac-672422f573d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:90226002-1343-4f2a-96ac-672422f573d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 758ea1ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 758ea1ca on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:44b84adf-ab5e-4e71-b5be-38c731e9f409:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:44b84adf-ab5e-4e71-b5be-38c731e9f409:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 76bc2388 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 76bc2388 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 76bc2388 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3966 (Mon Oct 22 09:40:04 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3966 (Mon Oct 22 09:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f531ab48", "local_conf_timestamp": 3966, "host-ts": 3966}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3966 (Mon Oct 22 09:40:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3966 (Mon Oct 22 09:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b26294af", "local_conf_timestamp": 3966, "host-ts": 3966}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ef946b3c-ef4b-4192-83b5-8e22d763cb3a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ef946b3c-ef4b-4192-83b5-8e22d763cb3a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7e228d4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7e228d4c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7e228d4c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0a39bc5b", "local_conf_timestamp": 3976, "host-ts": 3976}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1d1ff876-e5dc-44ce-8cb1-a1d3b0dc8fca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1d1ff876-e5dc-44ce-8cb1-a1d3b0dc8fca:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8158654a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8158654a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8158654a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3986 (Mon Oct 22 09:40:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3986 (Mon Oct 22 09:40:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9c0135b9", "local_conf_timestamp": 3986, "host-ts": 3986}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b615ebf6-3c87-42e5-b41a-abba1e9300de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b615ebf6-3c87-42e5-b41a-abba1e9300de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 84513862 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 84513862 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 84513862 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3986 (Mon Oct 22 09:40:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3986 (Mon Oct 22 09:40:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9c0135b9", "local_conf_timestamp": 3986, "host-ts": 3986}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Mon Oct 22 09:40:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Mon Oct 22 09:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5824d6b4-3613-4f6f-8bc8-cf546e7835d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5824d6b4-3613-4f6f-8bc8-cf546e7835d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 876b4736 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 876b4736 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 876b4736 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Mon Oct 22 09:40:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32320b7b", "local_conf_timestamp": 3996, "host-ts": 3996}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Mon Oct 22 09:40:33 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fab8a404", "local_conf_timestamp": 3996, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4d532bef-d9eb-4be6-809b-92f139b34f29:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4d532bef-d9eb-4be6-809b-92f139b34f29:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a77a6f4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8a77a6f4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8a77a6f4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Mon Oct 22 09:40:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32320b7b", "local_conf_timestamp": 3996, "host-ts": 3996}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Mon Oct 22 09:40:33 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3996 (Mon Oct 22 09:40:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fab8a404", "local_conf_timestamp": 3996, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:78d28bd8-9299-494b-9a81-b4ebb72b027a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:78d28bd8-9299-494b-9a81-b4ebb72b027a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8da5da12 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8da5da12 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8da5da12 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b8edaf2", "local_conf_timestamp": 4006, "host-ts": 4006}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3b969c76-112a-4f6f-b19d-a0a7e8d68880:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3b969c76-112a-4f6f-b19d-a0a7e8d68880:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 90ef0c02 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 90ef0c02 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 90ef0c02 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b8edaf2", "local_conf_timestamp": 4006, "host-ts": 4006}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Mon Oct 22 09:40:44 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Mon Oct 22 09:40:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:25e9ad5a-579e-4679-ba63-f8643bd3fc6c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:25e9ad5a-579e-4679-ba63-f8643bd3fc6c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 93f402ea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 93f402ea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 93f402ea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4017 (Mon Oct 22 09:40:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f2bad4a6", "local_conf_timestamp": 4017, "host-ts": 4016}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4016 (Mon Oct 22 09:40:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4468f13b", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:13ecc994-5878-48d3-8fda-421414522abd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:13ecc994-5878-48d3-8fda-421414522abd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 974b83d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 974b83d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 974b83d2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4026 (Mon Oct 22 09:41:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4026 (Mon Oct 22 09:41:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac99a137", "local_conf_timestamp": 4026, "host-ts": 4026}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4016 (Mon Oct 22 09:40:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4468f13b", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8d7a9dd6-fd6b-4150-84ca-fd153ed758c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8d7a9dd6-fd6b-4150-84ca-fd153ed758c7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9b03dcea on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9b03dcea on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9b03dcea on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4026 (Mon Oct 22 09:41:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4026 (Mon Oct 22 09:41:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ac99a137", "local_conf_timestamp": 4026, "host-ts": 4026}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Mon Oct 22 09:40:54 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4016 (Mon Oct 22 09:40:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4468f13b", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2a402aa9-c8da-460c-96f7-432fbaa0bb00:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2a402aa9-c8da-460c-96f7-432fbaa0bb00:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9e86bee6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9e86bee6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9e86bee6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Mon Oct 22 09:41:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Mon Oct 22 09:41:13 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4ce50f31", "local_conf_timestamp": 4036, "host-ts": 4036}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bf2431c7-bfac-4d73-b109-abfcc141f0e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bf2431c7-bfac-4d73-b109-abfcc141f0e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a15f7c3e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a15f7c3e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a15f7c3e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Mon Oct 22 09:41:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "02aa9ff5", "local_conf_timestamp": 4036, "host-ts": 4036}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Mon Oct 22 09:41:13 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4036 (Mon Oct 22 09:41:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4ce50f31", "local_conf_timestamp": 4036, "host-ts": 4036}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:82edd5bb-2ebf-40c6-9c4c-74d07254be34:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:82edd5bb-2ebf-40c6-9c4c-74d07254be34:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a45b661e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a45b661e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a45b661e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1369e16e-1a67-4f9b-bb7e-0388c32e41c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1369e16e-1a67-4f9b-bb7e-0388c32e41c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a572a774 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a572a774 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a572a774 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149c74b2", "local_conf_timestamp": 4046, "host-ts": 4046}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Mon Oct 22 09:41:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4046 (Mon Oct 22 09:41:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/425/ Build Number: 425 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #424 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #425 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/426/ Build Number: 426 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #424 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #425 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #426 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running c8f37666 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ca0334b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command ca0334b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb0fb252 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d23c358c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Tue Oct 23 01:41:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Tue Oct 23 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ef852cd", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d5b4749a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3993 (Tue Oct 23 01:41:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3993 (Tue Oct 23 01:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "efd6acb1", "local_conf_timestamp": 3993, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3986 (Tue Oct 23 01:41:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3986 (Tue Oct 23 01:41:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89c36e99", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false} lago.ssh: DEBUG: start task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d926eb44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dc6f44fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running df58251e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} lago.ssh: DEBUG: start task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e24389c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} lago.ssh: DEBUG: start task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e53a2a54 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e840bfce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running eb3563e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false} lago.ssh: DEBUG: start task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee849d4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false} lago.ssh: DEBUG: start task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f1a6f736 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false} lago.ssh: DEBUG: start task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f2cbe50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false} lago.ssh: DEBUG: start task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f9f3fdda on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4053 (Tue Oct 23 01:42:57 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4053 (Tue Oct 23 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_missing", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8b49ef54", "local_conf_timestamp": 4053, "host-ts": 4053}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Tue Oct 23 01:42:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4046 (Tue Oct 23 01:42:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running c8f37666 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ca0334b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command ca0334b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb0fb252 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d23c358c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Tue Oct 23 01:41:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Tue Oct 23 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ef852cd", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d5b4749a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3993 (Tue Oct 23 01:41:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3993 (Tue Oct 23 01:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "efd6acb1", "local_conf_timestamp": 3993, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3986 (Tue Oct 23 01:41:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3986 (Tue Oct 23 01:41:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89c36e99", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d926eb44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dc6f44fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running df58251e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e24389c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e53a2a54 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e840bfce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running eb3563e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee849d4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f1a6f736 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f2cbe50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f9f3fdda on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4053 (Tue Oct 23 01:42:57 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4053 (Tue Oct 23 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_missing", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8b49ef54", "local_conf_timestamp": 4053, "host-ts": 4053}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Tue Oct 23 01:42:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4046 (Tue Oct 23 01:42:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/427/ Build Number: 427 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #424 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #425 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #426 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #427 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/428/ Build Number: 428 Build Status: Fixed Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #424 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #425 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #426 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #427 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #428 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version [Evgheni Dereveanchin] Vacuum journal instead of deleting it ----------------- Failed Tests: ----------------- All tests passed
participants (1)
-
jenkins＠jenkins.phx.ovirt.org