[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 426 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/426/ Build Number: 426 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #411 [Galit] Update the config for poll u/s sources Changes for Build #412 [Galit] Update the config for poll u/s sources Changes for Build #413 [Galit] Update the config for poll u/s sources Changes for Build #414 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #415 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #416 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #417 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #418 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #419 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #420 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #421 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #422 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #423 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #424 [Galit] Fix stdci.yaml to be able to run poll upstream sources Changes for Build #425 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version Changes for Build #426 [Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running c8f37666 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ca0334b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command ca0334b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cb0fb252 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d23c358c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Tue Oct 23 01:41:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Tue Oct 23 01:41:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ef852cd", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false} lago.ssh: DEBUG: start task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d5b4749a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3993 (Tue Oct 23 01:41:56 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3993 (Tue Oct 23 01:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "efd6acb1", "local_conf_timestamp": 3993, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3986 (Tue Oct 23 01:41:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3986 (Tue Oct 23 01:41:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89c36e99", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false} lago.ssh: DEBUG: start task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d926eb44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dc6f44fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false} lago.ssh: DEBUG: start task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running df58251e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} lago.ssh: DEBUG: start task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e24389c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false} lago.ssh: DEBUG: start task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e53a2a54 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e840bfce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false} lago.ssh: DEBUG: start task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running eb3563e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false} lago.ssh: DEBUG: start task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee849d4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false} lago.ssh: DEBUG: start task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f1a6f736 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false} lago.ssh: DEBUG: start task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f2cbe50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false} lago.ssh: DEBUG: start task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f9f3fdda on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4053 (Tue Oct 23 01:42:57 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4053 (Tue Oct 23 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_missing", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8b49ef54", "local_conf_timestamp": 4053, "host-ts": 4053}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4046 (Tue Oct 23 01:42:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4046 (Tue Oct 23 01:42:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:036f236a-747e-435b-9519-3b9056fb86e9:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running c8f37666 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command c8f37666 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ae9c632f-52c7-4fc0-9ca4-61f9d16a95fd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ca0334b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command ca0334b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e33ba2b9-b327-47ac-bbc1-4b54402c1b75:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cb0fb252 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cb0fb252 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Tue Oct 23 01:41:36 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3972 (Tue Oct 23 01:41:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "81f2210c", "local_conf_timestamp": 3972, "host-ts": 3972}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:01df6e23-4b00-438c-b769-355c549bbbff:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d23c358c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d23c358c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Tue Oct 23 01:41:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Tue Oct 23 01:41:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3ef852cd", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3976 (Tue Oct 23 01:41:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3976 (Tue Oct 23 01:41:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b6246baa", "local_conf_timestamp": 3976, "host-ts": 3976}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3503dfed-99a8-4072-b072-1dee08e5846a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d5b4749a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d5b4749a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3993 (Tue Oct 23 01:41:56 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3993 (Tue Oct 23 01:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "efd6acb1", "local_conf_timestamp": 3993, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3986 (Tue Oct 23 01:41:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3986 (Tue Oct 23 01:41:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "89c36e99", "local_conf_timestamp": 3986, "host-ts": 3986}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:84cdb5a6-e545-4236-9967-241727d9a954:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d926eb44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d926eb44 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5e30746a-1486-4f5e-98ee-ada49e5a43f0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dc6f44fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dc6f44fe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4003 (Tue Oct 23 01:42:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4003 (Tue Oct 23 01:42:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f66a7d38", "local_conf_timestamp": 4003, "host-ts": 4003}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3996 (Tue Oct 23 01:42:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3997 (Tue Oct 23 01:42:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8dbf9492", "local_conf_timestamp": 3997, "host-ts": 3996}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7f7addcb-298d-4aad-a3fe-7e90c4655335:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running df58251e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command df58251e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:84fe5dbd-6035-42b3-997f-08311138360d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e24389c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e24389c6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4013 (Tue Oct 23 01:42:16 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4013 (Tue Oct 23 01:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "585943fa", "local_conf_timestamp": 4013, "host-ts": 4013}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4006 (Tue Oct 23 01:42:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4006 (Tue Oct 23 01:42:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f30ad6b0", "local_conf_timestamp": 4006, "host-ts": 4006}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cb8e14e0-cc6a-4945-9838-d9b20372ea4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e53a2a54 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e53a2a54 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:29fc9a08-ede0-4f6d-8332-5d81f473b4c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e840bfce on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e840bfce on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4023 (Tue Oct 23 01:42:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4023 (Tue Oct 23 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "717d06fd", "local_conf_timestamp": 4023, "host-ts": 4023}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4016 (Tue Oct 23 01:42:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4016 (Tue Oct 23 01:42:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f74c29b5", "local_conf_timestamp": 4016, "host-ts": 4016}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:205cedd2-155b-4757-80ed-08ac5b14cea4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running eb3563e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command eb3563e2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2fe8e1da-7987-4bb3-8bd2-a653f74bcddb:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee849d4c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ee849d4c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4033 (Tue Oct 23 01:42:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4034 (Tue Oct 23 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "412aad9c", "local_conf_timestamp": 4034, "host-ts": 4033}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4026 (Tue Oct 23 01:42:29 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4026 (Tue Oct 23 01:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "48a3f034", "local_conf_timestamp": 4026, "host-ts": 4026}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dd636571-4627-4381-9614-6acbf68372dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f1a6f736 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f1a6f736 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:66fe2771-15ac-4891-9081-b6685332f44d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f2cbe50e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f2cbe50e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4043 (Tue Oct 23 01:42:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4043 (Tue Oct 23 01:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6f91a11f", "local_conf_timestamp": 4043, "host-ts": 4043}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4036 (Tue Oct 23 01:42:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4037 (Tue Oct 23 01:42:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3be23fa7", "local_conf_timestamp": 4037, "host-ts": 4036}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:52250016-0bb6-4eea-8bd3-286cb804d5a8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f9f3fdda on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f9f3fdda on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4053 (Tue Oct 23 01:42:57 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4053 (Tue Oct 23 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_missing", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8b49ef54", "local_conf_timestamp": 4053, "host-ts": 4053}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4046 (Tue Oct 23 01:42:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4046 (Tue Oct 23 01:42:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "51ba673b", "local_conf_timestamp": 4046, "host-ts": 4046}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'