[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 401 - Failure!
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/401/ Build Number: 401 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... root: INFO: * Performing Activation... root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 79, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\nroot: INFO: * Performing Activation...\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/402/ Build Number: 402 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:b68b9515-30f1-44db-bed6-44d2738b2dc0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:b68b9515-30f1-44db-bed6-44d2738b2dc0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 4d9989d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4d9989d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 4d9989d2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3909 (Mon Oct 15 01:40:22 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3909 (Mon Oct 15 01:40:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7bca6ff6", "local_conf_timestamp": 3909, "host-ts": 3909}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3912 (Mon Oct 15 01:40:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3912 (Mon Oct 15 01:40:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b545216f", "local_conf_timestamp": 3912, "host-ts": 3912}, "global_maintenance": false} lago.ssh: DEBUG: start task:bfd3d8d3-2c78-4620-a38c-d200f3e30c71:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bfd3d8d3-2c78-4620-a38c-d200f3e30c71:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4ea68cbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 4ea68cbc on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:23e0748f-ee9b-4a24-8678-bcf6b6fbb52b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:23e0748f-ee9b-4a24-8678-bcf6b6fbb52b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4fc7dd58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4fc7dd58 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4fc7dd58 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3919 (Mon Oct 15 01:40:33 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3919 (Mon Oct 15 01:40:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a7ad2fc9", "local_conf_timestamp": 3919, "host-ts": 3919}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3912 (Mon Oct 15 01:40:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3912 (Mon Oct 15 01:40:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b545216f", "local_conf_timestamp": 3912, "host-ts": 3912}, "global_maintenance": false} lago.ssh: DEBUG: start task:ae003eae-0b79-47d3-bd44-a4887bf18f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ae003eae-0b79-47d3-bd44-a4887bf18f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 57f5fc26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 57f5fc26 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 57f5fc26 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3929 (Mon Oct 15 01:40:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3929 (Mon Oct 15 01:40:42 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3b1abee5", "local_conf_timestamp": 3929, "host-ts": 3929}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3922 (Mon Oct 15 01:40:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3922 (Mon Oct 15 01:40:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b98e2060", "local_conf_timestamp": 3922, "host-ts": 3922}, "global_maintenance": false} lago.ssh: DEBUG: start task:4fd20f7f-1e7b-47da-81a8-c4ed724935bd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4fd20f7f-1e7b-47da-81a8-c4ed724935bd:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5ba16fd6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5ba16fd6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5ba16fd6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3939 (Mon Oct 15 01:40:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3939 (Mon Oct 15 01:40:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "79b77b76", "local_conf_timestamp": 3939, "host-ts": 3939}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3932 (Mon Oct 15 01:40:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3932 (Mon Oct 15 01:40:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bdc8df65", "local_conf_timestamp": 3932, "host-ts": 3932}, "global_maintenance": false} lago.ssh: DEBUG: start task:d1201ae0-4524-4376-a31a-4bfbc3216d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d1201ae0-4524-4376-a31a-4bfbc3216d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5eb69df4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5eb69df4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5eb69df4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3939 (Mon Oct 15 01:40:52 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3939 (Mon Oct 15 01:40:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "79b77b76", "local_conf_timestamp": 3939, "host-ts": 3939}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3932 (Mon Oct 15 01:40:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3932 (Mon Oct 15 01:40:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bdc8df65", "local_conf_timestamp": 3932, "host-ts": 3932}, "global_maintenance": false} lago.ssh: DEBUG: start task:33986491-d513-4366-9ecb-d36cf85ec530:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:33986491-d513-4366-9ecb-d36cf85ec530:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6257a962 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6257a962 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6257a962 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3949 (Mon Oct 15 01:41:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3950 (Mon Oct 15 01:41:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e50b465f", "local_conf_timestamp": 3950, "host-ts": 3949}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3942 (Mon Oct 15 01:40:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3942 (Mon Oct 15 01:40:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d72517e6", "local_conf_timestamp": 3942, "host-ts": 3942}, "global_maintenance": false} lago.ssh: DEBUG: start task:9f41633d-f4be-4cff-be69-575033c3f84b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9f41633d-f4be-4cff-be69-575033c3f84b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 654d6652 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 654d6652 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 654d6652 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3949 (Mon Oct 15 01:41:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3950 (Mon Oct 15 01:41:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e50b465f", "local_conf_timestamp": 3950, "host-ts": 3949}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3942 (Mon Oct 15 01:40:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3942 (Mon Oct 15 01:40:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d72517e6", "local_conf_timestamp": 3942, "host-ts": 3942}, "global_maintenance": false} lago.ssh: DEBUG: start task:87d1798f-da36-4bf1-ac0a-c017bcabd8e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:87d1798f-da36-4bf1-ac0a-c017bcabd8e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 686cc30a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 686cc30a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 686cc30a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3959 (Mon Oct 15 01:41:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3959 (Mon Oct 15 01:41:12 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2bfff178", "local_conf_timestamp": 3959, "host-ts": 3959}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3952 (Mon Oct 15 01:41:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3952 (Mon Oct 15 01:41:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d363e8e3", "local_conf_timestamp": 3952, "host-ts": 3952}, "global_maintenance": false} lago.ssh: DEBUG: start task:95fbec13-41a2-4817-a0c0-76cd14dcae82:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:95fbec13-41a2-4817-a0c0-76cd14dcae82:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6b55ae38 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6b55ae38 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6b55ae38 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3959 (Mon Oct 15 01:41:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3959 (Mon Oct 15 01:41:12 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2bfff178", "local_conf_timestamp": 3959, "host-ts": 3959}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3962 (Mon Oct 15 01:41:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3962 (Mon Oct 15 01:41:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c8c3162", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false} lago.ssh: DEBUG: start task:362674eb-acfb-4586-960b-0bd74a07e9d4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:362674eb-acfb-4586-960b-0bd74a07e9d4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6eaaac8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6eaaac8c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6eaaac8c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3969 (Mon Oct 15 01:41:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3970 (Mon Oct 15 01:41:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "621c3d9a", "local_conf_timestamp": 3970, "host-ts": 3969}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3962 (Mon Oct 15 01:41:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3962 (Mon Oct 15 01:41:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c8c3162", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false} lago.ssh: DEBUG: start task:a648568d-8a9b-4b06-bea1-f62b69c71a61:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a648568d-8a9b-4b06-bea1-f62b69c71a61:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71b5d1fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71b5d1fe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71b5d1fe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3979 (Mon Oct 15 01:41:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3979 (Mon Oct 15 01:41:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ace88abd", "local_conf_timestamp": 3979, "host-ts": 3979}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Mon Oct 15 01:41:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3972 (Mon Oct 15 01:41:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "68cace67", "local_conf_timestamp": 3972, "host-ts": 3972}, "global_maintenance": false} lago.ssh: DEBUG: start task:f2d8227b-a46b-49aa-ada5-cee44bed73f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f2d8227b-a46b-49aa-ada5-cee44bed73f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 74e170b8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 74e170b8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 74e170b8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3979 (Mon Oct 15 01:41:32 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3979 (Mon Oct 15 01:41:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ace88abd", "local_conf_timestamp": 3979, "host-ts": 3979}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3972 (Mon Oct 15 01:41:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3972 (Mon Oct 15 01:41:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "68cace67", "local_conf_timestamp": 3972, "host-ts": 3972}, "global_maintenance": false} lago.ssh: DEBUG: start task:9c6da9f4-7403-4ab5-9ead-2396ecce2955:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9c6da9f4-7403-4ab5-9ead-2396ecce2955:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 780d5680 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 780d5680 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 780d5680 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3989 (Mon Oct 15 01:41:42 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3989 (Mon Oct 15 01:41:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a22b3da5", "local_conf_timestamp": 3989, "host-ts": 3989}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3982 (Mon Oct 15 01:41:34 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3982 (Mon Oct 15 01:41:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "572dcb54", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false} lago.ssh: DEBUG: start task:92836948-f4f4-49cb-a0a0-07c451a9bda0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:92836948-f4f4-49cb-a0a0-07c451a9bda0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 79213b90 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 79213b90 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 79213b90 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3989 (Mon Oct 15 01:41:42 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3989 (Mon Oct 15 01:41:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a22b3da5", "local_conf_timestamp": 3989, "host-ts": 3989}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3982 (Mon Oct 15 01:41:34 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3982 (Mon Oct 15 01:41:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "572dcb54", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false} lago.ssh: DEBUG: start task:354f9bec-0ba3-443e-b556-7609d0951714:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:354f9bec-0ba3-443e-b556-7609d0951714:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 803f1398 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 803f1398 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 803f1398 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3999 (Mon Oct 15 01:41:52 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=3999 (Mon Oct 15 01:41:52 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_missing", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "ac90fcbc", "local_conf_timestamp": 3999, "host-ts": 3999}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3992 (Mon Oct 15 01:41:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3992 (Mon Oct 15 01:41:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7ebbf29f", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:b68b9515-30f1-44db-bed6-44d2738b2dc0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:b68b9515-30f1-44db-bed6-44d2738b2dc0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 4d9989d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4d9989d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 4d9989d2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3909 (Mon Oct 15 01:40:22 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3909 (Mon Oct 15 01:40:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7bca6ff6", "local_conf_timestamp": 3909, "host-ts": 3909}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3912 (Mon Oct 15 01:40:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3912 (Mon Oct 15 01:40:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b545216f", "local_conf_timestamp": 3912, "host-ts": 3912}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bfd3d8d3-2c78-4620-a38c-d200f3e30c71:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bfd3d8d3-2c78-4620-a38c-d200f3e30c71:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4ea68cbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 4ea68cbc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:23e0748f-ee9b-4a24-8678-bcf6b6fbb52b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:23e0748f-ee9b-4a24-8678-bcf6b6fbb52b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4fc7dd58 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4fc7dd58 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4fc7dd58 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3919 (Mon Oct 15 01:40:33 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3919 (Mon Oct 15 01:40:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a7ad2fc9", "local_conf_timestamp": 3919, "host-ts": 3919}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3912 (Mon Oct 15 01:40:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3912 (Mon Oct 15 01:40:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b545216f", "local_conf_timestamp": 3912, "host-ts": 3912}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ae003eae-0b79-47d3-bd44-a4887bf18f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ae003eae-0b79-47d3-bd44-a4887bf18f07:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 57f5fc26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 57f5fc26 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 57f5fc26 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3929 (Mon Oct 15 01:40:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3929 (Mon Oct 15 01:40:42 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3b1abee5", "local_conf_timestamp": 3929, "host-ts": 3929}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3922 (Mon Oct 15 01:40:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3922 (Mon Oct 15 01:40:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b98e2060", "local_conf_timestamp": 3922, "host-ts": 3922}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4fd20f7f-1e7b-47da-81a8-c4ed724935bd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4fd20f7f-1e7b-47da-81a8-c4ed724935bd:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5ba16fd6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5ba16fd6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5ba16fd6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3939 (Mon Oct 15 01:40:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3939 (Mon Oct 15 01:40:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "79b77b76", "local_conf_timestamp": 3939, "host-ts": 3939}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3932 (Mon Oct 15 01:40:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3932 (Mon Oct 15 01:40:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bdc8df65", "local_conf_timestamp": 3932, "host-ts": 3932}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d1201ae0-4524-4376-a31a-4bfbc3216d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d1201ae0-4524-4376-a31a-4bfbc3216d89:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5eb69df4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5eb69df4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5eb69df4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3939 (Mon Oct 15 01:40:52 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3939 (Mon Oct 15 01:40:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "79b77b76", "local_conf_timestamp": 3939, "host-ts": 3939}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3932 (Mon Oct 15 01:40:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3932 (Mon Oct 15 01:40:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bdc8df65", "local_conf_timestamp": 3932, "host-ts": 3932}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:33986491-d513-4366-9ecb-d36cf85ec530:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:33986491-d513-4366-9ecb-d36cf85ec530:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6257a962 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6257a962 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6257a962 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3949 (Mon Oct 15 01:41:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3950 (Mon Oct 15 01:41:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e50b465f", "local_conf_timestamp": 3950, "host-ts": 3949}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3942 (Mon Oct 15 01:40:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3942 (Mon Oct 15 01:40:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d72517e6", "local_conf_timestamp": 3942, "host-ts": 3942}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9f41633d-f4be-4cff-be69-575033c3f84b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9f41633d-f4be-4cff-be69-575033c3f84b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 654d6652 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 654d6652 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 654d6652 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3949 (Mon Oct 15 01:41:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3950 (Mon Oct 15 01:41:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e50b465f", "local_conf_timestamp": 3950, "host-ts": 3949}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3942 (Mon Oct 15 01:40:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3942 (Mon Oct 15 01:40:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d72517e6", "local_conf_timestamp": 3942, "host-ts": 3942}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:87d1798f-da36-4bf1-ac0a-c017bcabd8e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:87d1798f-da36-4bf1-ac0a-c017bcabd8e5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 686cc30a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 686cc30a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 686cc30a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3959 (Mon Oct 15 01:41:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3959 (Mon Oct 15 01:41:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2bfff178", "local_conf_timestamp": 3959, "host-ts": 3959}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3952 (Mon Oct 15 01:41:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3952 (Mon Oct 15 01:41:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d363e8e3", "local_conf_timestamp": 3952, "host-ts": 3952}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:95fbec13-41a2-4817-a0c0-76cd14dcae82:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:95fbec13-41a2-4817-a0c0-76cd14dcae82:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6b55ae38 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6b55ae38 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6b55ae38 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3959 (Mon Oct 15 01:41:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3959 (Mon Oct 15 01:41:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2bfff178", "local_conf_timestamp": 3959, "host-ts": 3959}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3962 (Mon Oct 15 01:41:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3962 (Mon Oct 15 01:41:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c8c3162", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:362674eb-acfb-4586-960b-0bd74a07e9d4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:362674eb-acfb-4586-960b-0bd74a07e9d4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6eaaac8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6eaaac8c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6eaaac8c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3969 (Mon Oct 15 01:41:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3970 (Mon Oct 15 01:41:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "621c3d9a", "local_conf_timestamp": 3970, "host-ts": 3969}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3962 (Mon Oct 15 01:41:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3962 (Mon Oct 15 01:41:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c8c3162", "local_conf_timestamp": 3962, "host-ts": 3962}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a648568d-8a9b-4b06-bea1-f62b69c71a61:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a648568d-8a9b-4b06-bea1-f62b69c71a61:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71b5d1fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71b5d1fe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71b5d1fe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3979 (Mon Oct 15 01:41:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3979 (Mon Oct 15 01:41:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ace88abd", "local_conf_timestamp": 3979, "host-ts": 3979}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Mon Oct 15 01:41:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3972 (Mon Oct 15 01:41:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "68cace67", "local_conf_timestamp": 3972, "host-ts": 3972}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f2d8227b-a46b-49aa-ada5-cee44bed73f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f2d8227b-a46b-49aa-ada5-cee44bed73f2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 74e170b8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 74e170b8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 74e170b8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3979 (Mon Oct 15 01:41:32 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3979 (Mon Oct 15 01:41:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ace88abd", "local_conf_timestamp": 3979, "host-ts": 3979}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3972 (Mon Oct 15 01:41:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3972 (Mon Oct 15 01:41:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "68cace67", "local_conf_timestamp": 3972, "host-ts": 3972}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9c6da9f4-7403-4ab5-9ead-2396ecce2955:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9c6da9f4-7403-4ab5-9ead-2396ecce2955:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 780d5680 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 780d5680 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 780d5680 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3989 (Mon Oct 15 01:41:42 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3989 (Mon Oct 15 01:41:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a22b3da5", "local_conf_timestamp": 3989, "host-ts": 3989}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3982 (Mon Oct 15 01:41:34 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3982 (Mon Oct 15 01:41:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "572dcb54", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:92836948-f4f4-49cb-a0a0-07c451a9bda0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:92836948-f4f4-49cb-a0a0-07c451a9bda0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 79213b90 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 79213b90 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 79213b90 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3989 (Mon Oct 15 01:41:42 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3989 (Mon Oct 15 01:41:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a22b3da5", "local_conf_timestamp": 3989, "host-ts": 3989}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3982 (Mon Oct 15 01:41:34 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3982 (Mon Oct 15 01:41:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "572dcb54", "local_conf_timestamp": 3982, "host-ts": 3982}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:354f9bec-0ba3-443e-b556-7609d0951714:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:354f9bec-0ba3-443e-b556-7609d0951714:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 803f1398 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 803f1398 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 803f1398 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3999 (Mon Oct 15 01:41:52 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=3999 (Mon Oct 15 01:41:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_missing", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "ac90fcbc", "local_conf_timestamp": 3999, "host-ts": 3999}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3992 (Mon Oct 15 01:41:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3992 (Mon Oct 15 01:41:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7ebbf29f", "local_conf_timestamp": 3992, "host-ts": 3992}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/403/ Build Number: 403 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/404/ Build Number: 404 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 65, in local_maintenance lambda: host_service.get().status == types.HostStatus.MAINTENANCE or File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 246, in assert_equals_within time.sleep(3) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/405/ Build Number: 405 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container Changes for Build #405 [Gal Ben Haim] Add run-ost-container ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:1ec740d6-f89f-4e3b-a076-a6da0ec1c4cf:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:1ec740d6-f89f-4e3b-a076-a6da0ec1c4cf:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running c0913fe2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c0913fe2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command c0913fe2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3866 (Tue Oct 16 01:42:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3866 (Tue Oct 16 01:42:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "438be135", "local_conf_timestamp": 3866, "host-ts": 3866}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3875 (Tue Oct 16 01:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3875 (Tue Oct 16 01:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb50ee11", "local_conf_timestamp": 3875, "host-ts": 3875}, "global_maintenance": false} lago.ssh: DEBUG: start task:13b836af-92aa-4bf0-af08-6d4ba978cd36:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:13b836af-92aa-4bf0-af08-6d4ba978cd36:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c1992a94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command c1992a94 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:a64a910e-e045-41f9-bc54-d1c37d3f468a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a64a910e-e045-41f9-bc54-d1c37d3f468a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c2d6c3bc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c2d6c3bc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c2d6c3bc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3886 (Tue Oct 16 01:42:36 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3886 (Tue Oct 16 01:42:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "91ab22e5", "local_conf_timestamp": 3886, "host-ts": 3886}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3885 (Tue Oct 16 01:42:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3886 (Tue Oct 16 01:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1dbeba98", "local_conf_timestamp": 3886, "host-ts": 3885}, "global_maintenance": false} lago.ssh: DEBUG: start task:b78db1ba-0d03-4a8d-8cc4-1e4e70fdcaef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b78db1ba-0d03-4a8d-8cc4-1e4e70fdcaef:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ca23dd94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ca23dd94 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ca23dd94 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3896 (Tue Oct 16 01:42:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3896 (Tue Oct 16 01:42:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47f69bd9", "local_conf_timestamp": 3896, "host-ts": 3896}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3895 (Tue Oct 16 01:42:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3895 (Tue Oct 16 01:42:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7cc21bf", "local_conf_timestamp": 3895, "host-ts": 3895}, "global_maintenance": false} lago.ssh: DEBUG: start task:1e0c5887-78f5-4675-9613-2014b5aba943:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1e0c5887-78f5-4675-9613-2014b5aba943:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running cd4eca6a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command cd4eca6a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command cd4eca6a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3896 (Tue Oct 16 01:42:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3896 (Tue Oct 16 01:42:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47f69bd9", "local_conf_timestamp": 3896, "host-ts": 3896}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3895 (Tue Oct 16 01:42:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3895 (Tue Oct 16 01:42:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7cc21bf", "local_conf_timestamp": 3895, "host-ts": 3895}, "global_maintenance": false} lago.ssh: DEBUG: start task:2ca5cf5b-a742-4a4d-a512-5480a908cbf6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2ca5cf5b-a742-4a4d-a512-5480a908cbf6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d07b6cc0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d07b6cc0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d07b6cc0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3906 (Tue Oct 16 01:42:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3907 (Tue Oct 16 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9ac8e0f8", "local_conf_timestamp": 3907, "host-ts": 3906}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3905 (Tue Oct 16 01:42:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3905 (Tue Oct 16 01:42:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1d394e2", "local_conf_timestamp": 3905, "host-ts": 3905}, "global_maintenance": false} lago.ssh: DEBUG: start task:b8e5dd8a-c0d0-48c5-9b90-100fb6039190:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b8e5dd8a-c0d0-48c5-9b90-100fb6039190:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d386fa56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d386fa56 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d386fa56 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3906 (Tue Oct 16 01:42:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3907 (Tue Oct 16 01:42:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9ac8e0f8", "local_conf_timestamp": 3907, "host-ts": 3906}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3905 (Tue Oct 16 01:42:55 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3905 (Tue Oct 16 01:42:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1d394e2", "local_conf_timestamp": 3905, "host-ts": 3905}, "global_maintenance": false} lago.ssh: DEBUG: start task:fa446718-d37e-4eee-85dc-77b2601449ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fa446718-d37e-4eee-85dc-77b2601449ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d685a108 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d685a108 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d685a108 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3916 (Tue Oct 16 01:43:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3916 (Tue Oct 16 01:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43fceeac", "local_conf_timestamp": 3916, "host-ts": 3916}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3915 (Tue Oct 16 01:43:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3915 (Tue Oct 16 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a5956be7", "local_conf_timestamp": 3915, "host-ts": 3915}, "global_maintenance": false} lago.ssh: DEBUG: start task:d73080cf-ad0b-4515-9c0d-0e0e741f387f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d73080cf-ad0b-4515-9c0d-0e0e741f387f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d96d42cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d96d42cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d96d42cc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3916 (Tue Oct 16 01:43:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3916 (Tue Oct 16 01:43:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43fceeac", "local_conf_timestamp": 3916, "host-ts": 3916}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3915 (Tue Oct 16 01:43:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3915 (Tue Oct 16 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a5956be7", "local_conf_timestamp": 3915, "host-ts": 3915}, "global_maintenance": false} lago.ssh: DEBUG: start task:753cc227-2c99-4f0c-a2de-17b1d9dae85f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:753cc227-2c99-4f0c-a2de-17b1d9dae85f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dc655028 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dc655028 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dc655028 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3926 (Tue Oct 16 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3926 (Tue Oct 16 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6ad8abab", "local_conf_timestamp": 3926, "host-ts": 3926}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3925 (Tue Oct 16 01:43:16 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3925 (Tue Oct 16 01:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1a7ab266", "local_conf_timestamp": 3925, "host-ts": 3925}, "global_maintenance": false} lago.ssh: DEBUG: start task:7f4385dd-9dd8-4645-a298-4596ba7bcace:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7f4385dd-9dd8-4645-a298-4596ba7bcace:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running df4dfc5e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command df4dfc5e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command df4dfc5e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3926 (Tue Oct 16 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3926 (Tue Oct 16 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6ad8abab", "local_conf_timestamp": 3926, "host-ts": 3926}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3925 (Tue Oct 16 01:43:16 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3925 (Tue Oct 16 01:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1a7ab266", "local_conf_timestamp": 3925, "host-ts": 3925}, "global_maintenance": false} lago.ssh: DEBUG: start task:feb6706c-3b99-4d6f-9216-140e1c4e96e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:feb6706c-3b99-4d6f-9216-140e1c4e96e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e23756cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e23756cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e23756cc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3936 (Tue Oct 16 01:43:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3937 (Tue Oct 16 01:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b3eca5ff", "local_conf_timestamp": 3937, "host-ts": 3936}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Tue Oct 16 01:43:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3935 (Tue Oct 16 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1e3c4d63", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:d44953e5-c2ab-418d-ba19-ef5042d9ba61:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d44953e5-c2ab-418d-ba19-ef5042d9ba61:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e5588074 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e5588074 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e5588074 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3936 (Tue Oct 16 01:43:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3937 (Tue Oct 16 01:43:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b3eca5ff", "local_conf_timestamp": 3937, "host-ts": 3936}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Tue Oct 16 01:43:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3935 (Tue Oct 16 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1e3c4d63", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false} lago.ssh: DEBUG: start task:026982ee-353a-4310-b385-dcfbaad58cb5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:026982ee-353a-4310-b385-dcfbaad58cb5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e85e042e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e85e042e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e85e042e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3946 (Tue Oct 16 01:43:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3947 (Tue Oct 16 01:43:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f971133", "local_conf_timestamp": 3947, "host-ts": 3946}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Tue Oct 16 01:43:35 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3945 (Tue Oct 16 01:43:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03ecb078", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:aba3d5dd-d944-4997-8a81-d511d818aab5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aba3d5dd-d944-4997-8a81-d511d818aab5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running eb38edee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command eb38edee on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command eb38edee on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3946 (Tue Oct 16 01:43:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3947 (Tue Oct 16 01:43:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f971133", "local_conf_timestamp": 3947, "host-ts": 3946}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Tue Oct 16 01:43:35 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3945 (Tue Oct 16 01:43:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03ecb078", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false} lago.ssh: DEBUG: start task:7e8de7f0-b7f8-4c73-8b85-0dd5b9a30a59:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7e8de7f0-b7f8-4c73-8b85-0dd5b9a30a59:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee0e7c6e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ee0e7c6e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ee0e7c6e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3956 (Tue Oct 16 01:43:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3956 (Tue Oct 16 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3f19fe2e", "local_conf_timestamp": 3956, "host-ts": 3956}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Tue Oct 16 01:43:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3955 (Tue Oct 16 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4c483c14", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false} lago.ssh: DEBUG: start task:f23e82f2-6e90-4362-9319-c699d721fb13:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f23e82f2-6e90-4362-9319-c699d721fb13:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ef1a232e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ef1a232e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ef1a232e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3956 (Tue Oct 16 01:43:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3956 (Tue Oct 16 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3f19fe2e", "local_conf_timestamp": 3956, "host-ts": 3956}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Tue Oct 16 01:43:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3955 (Tue Oct 16 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4c483c14", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:1ec740d6-f89f-4e3b-a076-a6da0ec1c4cf:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:1ec740d6-f89f-4e3b-a076-a6da0ec1c4cf:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running c0913fe2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c0913fe2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command c0913fe2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3866 (Tue Oct 16 01:42:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3866 (Tue Oct 16 01:42:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "438be135", "local_conf_timestamp": 3866, "host-ts": 3866}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3875 (Tue Oct 16 01:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3875 (Tue Oct 16 01:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb50ee11", "local_conf_timestamp": 3875, "host-ts": 3875}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:13b836af-92aa-4bf0-af08-6d4ba978cd36:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:13b836af-92aa-4bf0-af08-6d4ba978cd36:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c1992a94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command c1992a94 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:a64a910e-e045-41f9-bc54-d1c37d3f468a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a64a910e-e045-41f9-bc54-d1c37d3f468a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c2d6c3bc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c2d6c3bc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c2d6c3bc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3886 (Tue Oct 16 01:42:36 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3886 (Tue Oct 16 01:42:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "91ab22e5", "local_conf_timestamp": 3886, "host-ts": 3886}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3885 (Tue Oct 16 01:42:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3886 (Tue Oct 16 01:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1dbeba98", "local_conf_timestamp": 3886, "host-ts": 3885}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b78db1ba-0d03-4a8d-8cc4-1e4e70fdcaef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b78db1ba-0d03-4a8d-8cc4-1e4e70fdcaef:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ca23dd94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ca23dd94 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ca23dd94 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3896 (Tue Oct 16 01:42:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3896 (Tue Oct 16 01:42:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47f69bd9", "local_conf_timestamp": 3896, "host-ts": 3896}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3895 (Tue Oct 16 01:42:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3895 (Tue Oct 16 01:42:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7cc21bf", "local_conf_timestamp": 3895, "host-ts": 3895}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1e0c5887-78f5-4675-9613-2014b5aba943:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1e0c5887-78f5-4675-9613-2014b5aba943:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running cd4eca6a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command cd4eca6a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command cd4eca6a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3896 (Tue Oct 16 01:42:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3896 (Tue Oct 16 01:42:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "47f69bd9", "local_conf_timestamp": 3896, "host-ts": 3896}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3895 (Tue Oct 16 01:42:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3895 (Tue Oct 16 01:42:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7cc21bf", "local_conf_timestamp": 3895, "host-ts": 3895}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2ca5cf5b-a742-4a4d-a512-5480a908cbf6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2ca5cf5b-a742-4a4d-a512-5480a908cbf6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d07b6cc0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d07b6cc0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d07b6cc0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3906 (Tue Oct 16 01:42:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3907 (Tue Oct 16 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9ac8e0f8", "local_conf_timestamp": 3907, "host-ts": 3906}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3905 (Tue Oct 16 01:42:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3905 (Tue Oct 16 01:42:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1d394e2", "local_conf_timestamp": 3905, "host-ts": 3905}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b8e5dd8a-c0d0-48c5-9b90-100fb6039190:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b8e5dd8a-c0d0-48c5-9b90-100fb6039190:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d386fa56 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d386fa56 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d386fa56 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3906 (Tue Oct 16 01:42:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3907 (Tue Oct 16 01:42:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9ac8e0f8", "local_conf_timestamp": 3907, "host-ts": 3906}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3905 (Tue Oct 16 01:42:55 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3905 (Tue Oct 16 01:42:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1d394e2", "local_conf_timestamp": 3905, "host-ts": 3905}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fa446718-d37e-4eee-85dc-77b2601449ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fa446718-d37e-4eee-85dc-77b2601449ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d685a108 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d685a108 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d685a108 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3916 (Tue Oct 16 01:43:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3916 (Tue Oct 16 01:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43fceeac", "local_conf_timestamp": 3916, "host-ts": 3916}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3915 (Tue Oct 16 01:43:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3915 (Tue Oct 16 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a5956be7", "local_conf_timestamp": 3915, "host-ts": 3915}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d73080cf-ad0b-4515-9c0d-0e0e741f387f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d73080cf-ad0b-4515-9c0d-0e0e741f387f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d96d42cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d96d42cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d96d42cc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3916 (Tue Oct 16 01:43:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3916 (Tue Oct 16 01:43:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "43fceeac", "local_conf_timestamp": 3916, "host-ts": 3916}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3915 (Tue Oct 16 01:43:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3915 (Tue Oct 16 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a5956be7", "local_conf_timestamp": 3915, "host-ts": 3915}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:753cc227-2c99-4f0c-a2de-17b1d9dae85f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:753cc227-2c99-4f0c-a2de-17b1d9dae85f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dc655028 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dc655028 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dc655028 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3926 (Tue Oct 16 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3926 (Tue Oct 16 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6ad8abab", "local_conf_timestamp": 3926, "host-ts": 3926}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3925 (Tue Oct 16 01:43:16 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3925 (Tue Oct 16 01:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1a7ab266", "local_conf_timestamp": 3925, "host-ts": 3925}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7f4385dd-9dd8-4645-a298-4596ba7bcace:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7f4385dd-9dd8-4645-a298-4596ba7bcace:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running df4dfc5e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command df4dfc5e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command df4dfc5e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3926 (Tue Oct 16 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3926 (Tue Oct 16 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6ad8abab", "local_conf_timestamp": 3926, "host-ts": 3926}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3925 (Tue Oct 16 01:43:16 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3925 (Tue Oct 16 01:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1a7ab266", "local_conf_timestamp": 3925, "host-ts": 3925}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:feb6706c-3b99-4d6f-9216-140e1c4e96e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:feb6706c-3b99-4d6f-9216-140e1c4e96e9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e23756cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e23756cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e23756cc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3936 (Tue Oct 16 01:43:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3937 (Tue Oct 16 01:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b3eca5ff", "local_conf_timestamp": 3937, "host-ts": 3936}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Tue Oct 16 01:43:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3935 (Tue Oct 16 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1e3c4d63", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d44953e5-c2ab-418d-ba19-ef5042d9ba61:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d44953e5-c2ab-418d-ba19-ef5042d9ba61:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e5588074 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e5588074 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e5588074 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3936 (Tue Oct 16 01:43:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3937 (Tue Oct 16 01:43:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b3eca5ff", "local_conf_timestamp": 3937, "host-ts": 3936}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Tue Oct 16 01:43:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3935 (Tue Oct 16 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1e3c4d63", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:026982ee-353a-4310-b385-dcfbaad58cb5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:026982ee-353a-4310-b385-dcfbaad58cb5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e85e042e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e85e042e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e85e042e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3946 (Tue Oct 16 01:43:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3947 (Tue Oct 16 01:43:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f971133", "local_conf_timestamp": 3947, "host-ts": 3946}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Tue Oct 16 01:43:35 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3945 (Tue Oct 16 01:43:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03ecb078", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aba3d5dd-d944-4997-8a81-d511d818aab5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aba3d5dd-d944-4997-8a81-d511d818aab5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running eb38edee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command eb38edee on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command eb38edee on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3946 (Tue Oct 16 01:43:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3947 (Tue Oct 16 01:43:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f971133", "local_conf_timestamp": 3947, "host-ts": 3946}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Tue Oct 16 01:43:35 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3945 (Tue Oct 16 01:43:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "03ecb078", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7e8de7f0-b7f8-4c73-8b85-0dd5b9a30a59:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7e8de7f0-b7f8-4c73-8b85-0dd5b9a30a59:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee0e7c6e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ee0e7c6e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ee0e7c6e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3956 (Tue Oct 16 01:43:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3956 (Tue Oct 16 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3f19fe2e", "local_conf_timestamp": 3956, "host-ts": 3956}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Tue Oct 16 01:43:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3955 (Tue Oct 16 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4c483c14", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f23e82f2-6e90-4362-9319-c699d721fb13:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f23e82f2-6e90-4362-9319-c699d721fb13:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ef1a232e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ef1a232e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ef1a232e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3956 (Tue Oct 16 01:43:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3956 (Tue Oct 16 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3f19fe2e", "local_conf_timestamp": 3956, "host-ts": 3956}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Tue Oct 16 01:43:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3955 (Tue Oct 16 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4c483c14", "local_conf_timestamp": 3955, "host-ts": 3955}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/406/ Build Number: 406 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container Changes for Build #405 [Gal Ben Haim] Add run-ost-container Changes for Build #406 [Gal Ben Haim] Add run-ost-container [Evgheni Dereveanchin] Remove fcraw mirror sync job ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.add_disk Error Message: 143 -------------------- >> begin captured logging << -------------------- cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 69, in wrapper return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/004_basic_sanity.py", line 111, in add_disk lambda: File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/407/ Build Number: 407 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container Changes for Build #405 [Gal Ben Haim] Add run-ost-container Changes for Build #406 [Gal Ben Haim] Add run-ost-container [Evgheni Dereveanchin] Remove fcraw mirror sync job Changes for Build #407 [Gal Ben Haim] Add run-ost-container [Dafna Ron] Removed ovirt-provider-ovn v1 jobs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/408/ Build Number: 408 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container Changes for Build #405 [Gal Ben Haim] Add run-ost-container Changes for Build #406 [Gal Ben Haim] Add run-ost-container [Evgheni Dereveanchin] Remove fcraw mirror sync job Changes for Build #407 [Gal Ben Haim] Add run-ost-container [Dafna Ron] Removed ovirt-provider-ovn v1 jobs Changes for Build #408 [Gal Ben Haim] OST container improvements ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/409/ Build Number: 409 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container Changes for Build #405 [Gal Ben Haim] Add run-ost-container Changes for Build #406 [Gal Ben Haim] Add run-ost-container [Evgheni Dereveanchin] Remove fcraw mirror sync job Changes for Build #407 [Gal Ben Haim] Add run-ost-container [Dafna Ron] Removed ovirt-provider-ovn v1 jobs Changes for Build #408 [Gal Ben Haim] OST container improvements Changes for Build #409 [Gal Ben Haim] OST container improvements [Sandro Bonazzola] Revert "Removed ovirt-provider-ovn v1 jobs" ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:057752aa-35e7-4b85-b5ef-addebc75c5b3:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:057752aa-35e7-4b85-b5ef-addebc75c5b3:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 87d3da2c on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 87d3da2c on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 87d3da2c on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3881 (Wed Oct 17 09:39:19 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3881 (Wed Oct 17 09:39:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "12c7a981", "local_conf_timestamp": 3881, "host-ts": 3881}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3876 (Wed Oct 17 09:39:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3876 (Wed Oct 17 09:39:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c64ad9da", "local_conf_timestamp": 3876, "host-ts": 3876}, "global_maintenance": false} lago.ssh: DEBUG: start task:f31e6bd1-19e2-46c0-9b01-58be37a5b1b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f31e6bd1-19e2-46c0-9b01-58be37a5b1b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 88ec85d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 88ec85d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:a3bee47a-7f24-45c7-afda-b492dcb12b7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a3bee47a-7f24-45c7-afda-b492dcb12b7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a074248 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8a074248 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8a074248 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3881 (Wed Oct 17 09:39:19 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3881 (Wed Oct 17 09:39:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "12c7a981", "local_conf_timestamp": 3881, "host-ts": 3881}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3886 (Wed Oct 17 09:39:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3886 (Wed Oct 17 09:39:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f9addce9", "local_conf_timestamp": 3886, "host-ts": 3886}, "global_maintenance": false} lago.ssh: DEBUG: start task:d99ed652-3376-477e-ab81-cf00dcdb210b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d99ed652-3376-477e-ab81-cf00dcdb210b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 91115038 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 91115038 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 91115038 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3901 (Wed Oct 17 09:39:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3901 (Wed Oct 17 09:39:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ff36035", "local_conf_timestamp": 3901, "host-ts": 3901}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3896 (Wed Oct 17 09:39:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3896 (Wed Oct 17 09:39:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fdeb23ec", "local_conf_timestamp": 3896, "host-ts": 3896}, "global_maintenance": false} lago.ssh: DEBUG: start task:d320697f-f885-4783-817c-b2fa75a64385:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d320697f-f885-4783-817c-b2fa75a64385:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9446201c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9446201c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9446201c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3901 (Wed Oct 17 09:39:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3901 (Wed Oct 17 09:39:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ff36035", "local_conf_timestamp": 3901, "host-ts": 3901}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3896 (Wed Oct 17 09:39:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3896 (Wed Oct 17 09:39:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fdeb23ec", "local_conf_timestamp": 3896, "host-ts": 3896}, "global_maintenance": false} lago.ssh: DEBUG: start task:5c0be3ed-f852-4fc8-ba22-79dc9ddc6fd9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5c0be3ed-f852-4fc8-ba22-79dc9ddc6fd9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9733e5e8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9733e5e8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9733e5e8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3911 (Wed Oct 17 09:39:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3911 (Wed Oct 17 09:39:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "81c05ef7", "local_conf_timestamp": 3911, "host-ts": 3911}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3906 (Wed Oct 17 09:39:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3906 (Wed Oct 17 09:39:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dcc9a329", "local_conf_timestamp": 3906, "host-ts": 3906}, "global_maintenance": false} lago.ssh: DEBUG: start task:572cca72-4ed1-43d2-ac3f-360688d6e85c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:572cca72-4ed1-43d2-ac3f-360688d6e85c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9a1dc79c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9a1dc79c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9a1dc79c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3911 (Wed Oct 17 09:39:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3911 (Wed Oct 17 09:39:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "81c05ef7", "local_conf_timestamp": 3911, "host-ts": 3911}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3906 (Wed Oct 17 09:39:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3906 (Wed Oct 17 09:39:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dcc9a329", "local_conf_timestamp": 3906, "host-ts": 3906}, "global_maintenance": false} lago.ssh: DEBUG: start task:b424ae52-a6e2-4503-9307-ab80af7c11d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b424ae52-a6e2-4503-9307-ab80af7c11d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9d0dbf70 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9d0dbf70 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9d0dbf70 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3911 (Wed Oct 17 09:39:49 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3911 (Wed Oct 17 09:39:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "81c05ef7", "local_conf_timestamp": 3911, "host-ts": 3911}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3916 (Wed Oct 17 09:39:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3916 (Wed Oct 17 09:39:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d88f5c2c", "local_conf_timestamp": 3916, "host-ts": 3916}, "global_maintenance": false} lago.ssh: DEBUG: start task:8c50aed0-f059-41d9-afa6-7f059dd76007:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8c50aed0-f059-41d9-afa6-7f059dd76007:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a026a8f2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a026a8f2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a026a8f2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3921 (Wed Oct 17 09:40:00 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3922 (Wed Oct 17 09:40:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "31ed4a4a", "local_conf_timestamp": 3922, "host-ts": 3921}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3916 (Wed Oct 17 09:39:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3916 (Wed Oct 17 09:39:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d88f5c2c", "local_conf_timestamp": 3916, "host-ts": 3916}, "global_maintenance": false} lago.ssh: DEBUG: start task:bf7a8136-be10-4023-b9cc-fa6837346dee:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bf7a8136-be10-4023-b9cc-fa6837346dee:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a33cd192 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a33cd192 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a33cd192 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3931 (Wed Oct 17 09:40:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3931 (Wed Oct 17 09:40:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "06d72532", "local_conf_timestamp": 3931, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3926 (Wed Oct 17 09:40:04 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3926 (Wed Oct 17 09:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "676085ad", "local_conf_timestamp": 3926, "host-ts": 3926}, "global_maintenance": false} lago.ssh: DEBUG: start task:82b629e2-09f4-4321-9c55-8afbb15910a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:82b629e2-09f4-4321-9c55-8afbb15910a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a6a01934 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a6a01934 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a6a01934 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3931 (Wed Oct 17 09:40:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3931 (Wed Oct 17 09:40:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "06d72532", "local_conf_timestamp": 3931, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3926 (Wed Oct 17 09:40:04 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3926 (Wed Oct 17 09:40:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "676085ad", "local_conf_timestamp": 3926, "host-ts": 3926}, "global_maintenance": false} lago.ssh: DEBUG: start task:783ed088-c50c-40da-b9a5-0d81e269d484:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:783ed088-c50c-40da-b9a5-0d81e269d484:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a9aa9082 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a9aa9082 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a9aa9082 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3941 (Wed Oct 17 09:40:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3942 (Wed Oct 17 09:40:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63a5c044", "local_conf_timestamp": 3942, "host-ts": 3941}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3936 (Wed Oct 17 09:40:14 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3936 (Wed Oct 17 09:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63267aa8", "local_conf_timestamp": 3936, "host-ts": 3936}, "global_maintenance": false} lago.ssh: DEBUG: start task:a8d4bb44-0407-4ae5-b86c-e7b6ee37adc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a8d4bb44-0407-4ae5-b86c-e7b6ee37adc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running acb2c646 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command acb2c646 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command acb2c646 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3941 (Wed Oct 17 09:40:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3942 (Wed Oct 17 09:40:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63a5c044", "local_conf_timestamp": 3942, "host-ts": 3941}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3936 (Wed Oct 17 09:40:14 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3936 (Wed Oct 17 09:40:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63267aa8", "local_conf_timestamp": 3936, "host-ts": 3936}, "global_maintenance": false} lago.ssh: DEBUG: start task:930e4fe6-a51d-4ed5-9a2b-0d30c1406325:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:930e4fe6-a51d-4ed5-9a2b-0d30c1406325:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running afac868e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command afac868e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command afac868e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Wed Oct 17 09:40:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3951 (Wed Oct 17 09:40:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccb49254", "local_conf_timestamp": 3951, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3946 (Wed Oct 17 09:40:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3946 (Wed Oct 17 09:40:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7ef687b3", "local_conf_timestamp": 3946, "host-ts": 3946}, "global_maintenance": false} lago.ssh: DEBUG: start task:fff4cd78-4fd2-4bfa-99bf-cd4e3858100a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fff4cd78-4fd2-4bfa-99bf-cd4e3858100a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b0ee892a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b0ee892a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b0ee892a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3951 (Wed Oct 17 09:40:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3951 (Wed Oct 17 09:40:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccb49254", "local_conf_timestamp": 3951, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3946 (Wed Oct 17 09:40:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=3946 (Wed Oct 17 09:40:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7ef687b3", "local_conf_timestamp": 3946, "host-ts": 3946}, "global_maintenance": false} lago.ssh: DEBUG: start task:9ae51024-a39a-44a9-b913-8f2c4d5bb0c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9ae51024-a39a-44a9-b913-8f2c4d5bb0c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b808ddaa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b808ddaa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b808ddaa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3961 (Wed Oct 17 09:40:40 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=3961 (Wed Oct 17 09:40:40 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "4b55e0ce", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3956 (Wed Oct 17 09:40:34 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3956 (Wed Oct 17 09:40:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "44ef2a27", "local_conf_timestamp": 3956, "host-ts": 3956}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:057752aa-35e7-4b85-b5ef-addebc75c5b3:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:057752aa-35e7-4b85-b5ef-addebc75c5b3:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 87d3da2c on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 87d3da2c on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 87d3da2c on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3881 (Wed Oct 17 09:39:19 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3881 (Wed Oct 17 09:39:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "12c7a981", "local_conf_timestamp": 3881, "host-ts": 3881}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3876 (Wed Oct 17 09:39:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3876 (Wed Oct 17 09:39:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c64ad9da", "local_conf_timestamp": 3876, "host-ts": 3876}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f31e6bd1-19e2-46c0-9b01-58be37a5b1b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f31e6bd1-19e2-46c0-9b01-58be37a5b1b5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 88ec85d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 88ec85d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:a3bee47a-7f24-45c7-afda-b492dcb12b7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a3bee47a-7f24-45c7-afda-b492dcb12b7f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a074248 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8a074248 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8a074248 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3881 (Wed Oct 17 09:39:19 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3881 (Wed Oct 17 09:39:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "12c7a981", "local_conf_timestamp": 3881, "host-ts": 3881}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3886 (Wed Oct 17 09:39:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3886 (Wed Oct 17 09:39:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f9addce9", "local_conf_timestamp": 3886, "host-ts": 3886}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d99ed652-3376-477e-ab81-cf00dcdb210b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d99ed652-3376-477e-ab81-cf00dcdb210b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 91115038 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 91115038 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 91115038 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3901 (Wed Oct 17 09:39:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3901 (Wed Oct 17 09:39:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ff36035", "local_conf_timestamp": 3901, "host-ts": 3901}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3896 (Wed Oct 17 09:39:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3896 (Wed Oct 17 09:39:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fdeb23ec", "local_conf_timestamp": 3896, "host-ts": 3896}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d320697f-f885-4783-817c-b2fa75a64385:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d320697f-f885-4783-817c-b2fa75a64385:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9446201c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9446201c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9446201c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3901 (Wed Oct 17 09:39:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3901 (Wed Oct 17 09:39:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2ff36035", "local_conf_timestamp": 3901, "host-ts": 3901}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3896 (Wed Oct 17 09:39:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3896 (Wed Oct 17 09:39:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fdeb23ec", "local_conf_timestamp": 3896, "host-ts": 3896}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5c0be3ed-f852-4fc8-ba22-79dc9ddc6fd9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5c0be3ed-f852-4fc8-ba22-79dc9ddc6fd9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9733e5e8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9733e5e8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9733e5e8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3911 (Wed Oct 17 09:39:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3911 (Wed Oct 17 09:39:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "81c05ef7", "local_conf_timestamp": 3911, "host-ts": 3911}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3906 (Wed Oct 17 09:39:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3906 (Wed Oct 17 09:39:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dcc9a329", "local_conf_timestamp": 3906, "host-ts": 3906}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:572cca72-4ed1-43d2-ac3f-360688d6e85c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:572cca72-4ed1-43d2-ac3f-360688d6e85c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9a1dc79c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9a1dc79c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9a1dc79c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3911 (Wed Oct 17 09:39:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3911 (Wed Oct 17 09:39:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "81c05ef7", "local_conf_timestamp": 3911, "host-ts": 3911}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3906 (Wed Oct 17 09:39:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3906 (Wed Oct 17 09:39:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dcc9a329", "local_conf_timestamp": 3906, "host-ts": 3906}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b424ae52-a6e2-4503-9307-ab80af7c11d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b424ae52-a6e2-4503-9307-ab80af7c11d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9d0dbf70 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9d0dbf70 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9d0dbf70 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3911 (Wed Oct 17 09:39:49 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3911 (Wed Oct 17 09:39:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "81c05ef7", "local_conf_timestamp": 3911, "host-ts": 3911}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3916 (Wed Oct 17 09:39:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3916 (Wed Oct 17 09:39:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d88f5c2c", "local_conf_timestamp": 3916, "host-ts": 3916}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8c50aed0-f059-41d9-afa6-7f059dd76007:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8c50aed0-f059-41d9-afa6-7f059dd76007:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a026a8f2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a026a8f2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a026a8f2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3921 (Wed Oct 17 09:40:00 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3922 (Wed Oct 17 09:40:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "31ed4a4a", "local_conf_timestamp": 3922, "host-ts": 3921}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3916 (Wed Oct 17 09:39:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3916 (Wed Oct 17 09:39:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d88f5c2c", "local_conf_timestamp": 3916, "host-ts": 3916}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bf7a8136-be10-4023-b9cc-fa6837346dee:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bf7a8136-be10-4023-b9cc-fa6837346dee:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a33cd192 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a33cd192 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a33cd192 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3931 (Wed Oct 17 09:40:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3931 (Wed Oct 17 09:40:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "06d72532", "local_conf_timestamp": 3931, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3926 (Wed Oct 17 09:40:04 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3926 (Wed Oct 17 09:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "676085ad", "local_conf_timestamp": 3926, "host-ts": 3926}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:82b629e2-09f4-4321-9c55-8afbb15910a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:82b629e2-09f4-4321-9c55-8afbb15910a6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a6a01934 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a6a01934 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a6a01934 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3931 (Wed Oct 17 09:40:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3931 (Wed Oct 17 09:40:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "06d72532", "local_conf_timestamp": 3931, "host-ts": 3931}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3926 (Wed Oct 17 09:40:04 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3926 (Wed Oct 17 09:40:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "676085ad", "local_conf_timestamp": 3926, "host-ts": 3926}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:783ed088-c50c-40da-b9a5-0d81e269d484:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:783ed088-c50c-40da-b9a5-0d81e269d484:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a9aa9082 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a9aa9082 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a9aa9082 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3941 (Wed Oct 17 09:40:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3942 (Wed Oct 17 09:40:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63a5c044", "local_conf_timestamp": 3942, "host-ts": 3941}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3936 (Wed Oct 17 09:40:14 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3936 (Wed Oct 17 09:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63267aa8", "local_conf_timestamp": 3936, "host-ts": 3936}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a8d4bb44-0407-4ae5-b86c-e7b6ee37adc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a8d4bb44-0407-4ae5-b86c-e7b6ee37adc7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running acb2c646 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command acb2c646 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command acb2c646 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3941 (Wed Oct 17 09:40:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3942 (Wed Oct 17 09:40:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63a5c044", "local_conf_timestamp": 3942, "host-ts": 3941}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3936 (Wed Oct 17 09:40:14 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3936 (Wed Oct 17 09:40:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63267aa8", "local_conf_timestamp": 3936, "host-ts": 3936}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:930e4fe6-a51d-4ed5-9a2b-0d30c1406325:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:930e4fe6-a51d-4ed5-9a2b-0d30c1406325:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running afac868e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command afac868e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command afac868e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Wed Oct 17 09:40:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3951 (Wed Oct 17 09:40:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccb49254", "local_conf_timestamp": 3951, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3946 (Wed Oct 17 09:40:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3946 (Wed Oct 17 09:40:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7ef687b3", "local_conf_timestamp": 3946, "host-ts": 3946}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fff4cd78-4fd2-4bfa-99bf-cd4e3858100a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fff4cd78-4fd2-4bfa-99bf-cd4e3858100a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b0ee892a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b0ee892a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b0ee892a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3951 (Wed Oct 17 09:40:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3951 (Wed Oct 17 09:40:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccb49254", "local_conf_timestamp": 3951, "host-ts": 3951}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3946 (Wed Oct 17 09:40:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=3946 (Wed Oct 17 09:40:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7ef687b3", "local_conf_timestamp": 3946, "host-ts": 3946}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9ae51024-a39a-44a9-b913-8f2c4d5bb0c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9ae51024-a39a-44a9-b913-8f2c4d5bb0c5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b808ddaa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b808ddaa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b808ddaa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3961 (Wed Oct 17 09:40:40 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=3961 (Wed Oct 17 09:40:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "4b55e0ce", "local_conf_timestamp": 3961, "host-ts": 3961}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3956 (Wed Oct 17 09:40:34 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3956 (Wed Oct 17 09:40:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "44ef2a27", "local_conf_timestamp": 3956, "host-ts": 3956}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/410/ Build Number: 410 Build Status: Fixed Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #401 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #402 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #403 [Eitan Raviv] networking: connect NFS to engine over IPv6 Changes for Build #404 [Gal Ben Haim] Add run-ost-container Changes for Build #405 [Gal Ben Haim] Add run-ost-container Changes for Build #406 [Gal Ben Haim] Add run-ost-container [Evgheni Dereveanchin] Remove fcraw mirror sync job Changes for Build #407 [Gal Ben Haim] Add run-ost-container [Dafna Ron] Removed ovirt-provider-ovn v1 jobs Changes for Build #408 [Gal Ben Haim] OST container improvements Changes for Build #409 [Gal Ben Haim] OST container improvements [Sandro Bonazzola] Revert "Removed ovirt-provider-ovn v1 jobs" Changes for Build #410 [Galit] Update the config for poll u/s sources ----------------- Failed Tests: ----------------- All tests passed
participants (1)
-
jenkins＠jenkins.phx.ovirt.org