Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/301/ Build Number: 301 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:b568df2a-efba-4c11-bb66-94bab60200fc:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:b568df2a-efba-4c11-bb66-94bab60200fc:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running d8d533ea on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d8d533ea on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command d8d533ea on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4166 (Tue Sep 11 09:48:17 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4166 (Tue Sep 11 09:48:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8ab9ec16", "local_conf_timestamp": 4166, "host-ts": 4166}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4174 (Tue Sep 11 09:48:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4174 (Tue Sep 11 09:48:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "094a8b0e", "local_conf_timestamp": 4174, "host-ts": 4174}, "global_maintenance": false} lago.ssh: DEBUG: start task:d0e64905-3ca7-45b3-9862-52a16623824e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d0e64905-3ca7-45b3-9862-52a16623824e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d9e96b34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command d9e96b34 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:5f6e575f-5152-444d-99eb-00beea626fb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5f6e575f-5152-444d-99eb-00beea626fb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running db25c47a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command db25c47a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command db25c47a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4187 (Tue Sep 11 09:48:38 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4186 (Tue Sep 11 09:48:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9ed3ef7e", "local_conf_timestamp": 4186, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Tue Sep 11 09:48:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4184 (Tue Sep 11 09:48:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "36ad8e3d", "local_conf_timestamp": 4184, "host-ts": 4184}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in _wait_for_engine_migration testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:b568df2a-efba-4c11-bb66-94bab60200fc:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:b568df2a-efba-4c11-bb66-94bab60200fc:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running d8d533ea on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d8d533ea on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command d8d533ea on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4166 (Tue Sep 11 09:48:17 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4166 (Tue Sep 11 09:48:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8ab9ec16", "local_conf_timestamp": 4166, "host-ts": 4166}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4174 (Tue Sep 11 09:48:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4174 (Tue Sep 11 09:48:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "094a8b0e", "local_conf_timestamp": 4174, "host-ts": 4174}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d0e64905-3ca7-45b3-9862-52a16623824e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d0e64905-3ca7-45b3-9862-52a16623824e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d9e96b34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command d9e96b34 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:5f6e575f-5152-444d-99eb-00beea626fb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5f6e575f-5152-444d-99eb-00beea626fb4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running db25c47a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command db25c47a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command db25c47a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4187 (Tue Sep 11 09:48:38 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4186 (Tue Sep 11 09:48:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "9ed3ef7e", "local_conf_timestamp": 4186, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Tue Sep 11 09:48:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4184 (Tue Sep 11 09:48:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "36ad8e3d", "local_conf_timestamp": 4184, "host-ts": 4184}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/303/ Build Number: 303 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/305/ Build Number: 305 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:9d9e8dc5-3f92-44fa-acf7-71c866210045:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:9d9e8dc5-3f92-44fa-acf7-71c866210045:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 291925b2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 291925b2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 291925b2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Wed Sep 12 17:41:54 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4203 (Wed Sep 12 17:41:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "372f84c4", "local_conf_timestamp": 4203, "host-ts": 4203}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Wed Sep 12 17:41:56 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4206 (Wed Sep 12 17:41:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "80f9e1b9", "local_conf_timestamp": 4206, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:02edf3d0-3c34-4262-8b9e-3c8a78d0dc96:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:02edf3d0-3c34-4262-8b9e-3c8a78d0dc96:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2a581f8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 2a581f8c on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:665c45d5-74a7-4e29-b68c-2044db74249f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:665c45d5-74a7-4e29-b68c-2044db74249f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2b76e72c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2b76e72c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2b76e72c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Wed Sep 12 17:42:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4214 (Wed Sep 12 17:42:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "752c5158", "local_conf_timestamp": 4214, "host-ts": 4213}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Wed Sep 12 17:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4215 (Wed Sep 12 17:42:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1db64f06", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:9db09ee2-8a44-4206-a125-f263717738f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9db09ee2-8a44-4206-a125-f263717738f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 32d5a1b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 32d5a1b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 32d5a1b6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Wed Sep 12 17:42:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4223 (Wed Sep 12 17:42:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "77ff55d7", "local_conf_timestamp": 4223, "host-ts": 4223}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Wed Sep 12 17:42:05 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4215 (Wed Sep 12 17:42:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1db64f06", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:55195913-d8cb-4866-926e-042ae540d44c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:55195913-d8cb-4866-926e-042ae540d44c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 36822712 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 36822712 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 36822712 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Wed Sep 12 17:42:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4233 (Wed Sep 12 17:42:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34c7197b", "local_conf_timestamp": 4233, "host-ts": 4233}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Wed Sep 12 17:42:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4225 (Wed Sep 12 17:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:d784fa39-da7d-4e2b-bc29-5c4ccfc1510f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d784fa39-da7d-4e2b-bc29-5c4ccfc1510f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3972006e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3972006e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3972006e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Wed Sep 12 17:42:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4233 (Wed Sep 12 17:42:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34c7197b", "local_conf_timestamp": 4233, "host-ts": 4233}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Wed Sep 12 17:42:16 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4225 (Wed Sep 12 17:42:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:16d0a7de-b1f2-4116-91c3-80bd8592b430:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:16d0a7de-b1f2-4116-91c3-80bd8592b430:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3c902ffa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3c902ffa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3c902ffa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Wed Sep 12 17:42:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4243 (Wed Sep 12 17:42:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c8bcadb7", "local_conf_timestamp": 4243, "host-ts": 4243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Wed Sep 12 17:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4235 (Wed Sep 12 17:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "62068494", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:761b2a17-2dc8-4592-a8cd-5284fb7a8ea8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:761b2a17-2dc8-4592-a8cd-5284fb7a8ea8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3f968244 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3f968244 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3f968244 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Wed Sep 12 17:42:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4243 (Wed Sep 12 17:42:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c8bcadb7", "local_conf_timestamp": 4243, "host-ts": 4243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Wed Sep 12 17:42:26 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4235 (Wed Sep 12 17:42:26 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "62068494", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:f4cef96a-6372-4711-be92-4b3d0b68df80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f4cef96a-6372-4711-be92-4b3d0b68df80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 42b1d640 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 42b1d640 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 42b1d640 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Wed Sep 12 17:42:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4254 (Wed Sep 12 17:42:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f8eb06d6", "local_conf_timestamp": 4254, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Wed Sep 12 17:42:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Wed Sep 12 17:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e6df2835", "local_conf_timestamp": 4246, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:594d0f44-f3a9-422d-9360-7862752a3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:594d0f44-f3a9-422d-9360-7862752a3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 45cc9518 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 45cc9518 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 45cc9518 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Wed Sep 12 17:42:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4254 (Wed Sep 12 17:42:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f8eb06d6", "local_conf_timestamp": 4254, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Wed Sep 12 17:42:36 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4246 (Wed Sep 12 17:42:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e6df2835", "local_conf_timestamp": 4246, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:008a1a58-ce1d-4bf8-b33a-b572228a1c4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:008a1a58-ce1d-4bf8-b33a-b572228a1c4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 48c7eda8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 48c7eda8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 48c7eda8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Wed Sep 12 17:42:54 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4263 (Wed Sep 12 17:42:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4fabd672", "local_conf_timestamp": 4263, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Wed Sep 12 17:42:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Wed Sep 12 17:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} lago.ssh: DEBUG: start task:f7c2e934-99e9-47fc-88bb-310b327fa881:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f7c2e934-99e9-47fc-88bb-310b327fa881:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4bf1af3c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4bf1af3c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4bf1af3c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Wed Sep 12 17:42:54 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4263 (Wed Sep 12 17:42:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4fabd672", "local_conf_timestamp": 4263, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Wed Sep 12 17:42:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Wed Sep 12 17:42:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} lago.ssh: DEBUG: start task:3b21f3e7-2780-4f8a-9bd1-4332137f0abe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3b21f3e7-2780-4f8a-9bd1-4332137f0abe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4f0b5d8a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4f0b5d8a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4f0b5d8a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Wed Sep 12 17:43:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4273 (Wed Sep 12 17:43:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e198e8b0", "local_conf_timestamp": 4273, "host-ts": 4273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4265 (Wed Sep 12 17:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4265 (Wed Sep 12 17:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c47f5f0b", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false} lago.ssh: DEBUG: start task:57195e55-2edf-498b-9dd6-8f409872f39d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:57195e55-2edf-498b-9dd6-8f409872f39d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 52346bb4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 52346bb4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 52346bb4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4273 (Wed Sep 12 17:43:04 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4273 (Wed Sep 12 17:43:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e198e8b0", "local_conf_timestamp": 4273, "host-ts": 4273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4265 (Wed Sep 12 17:42:56 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4265 (Wed Sep 12 17:42:56 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c47f5f0b", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false} lago.ssh: DEBUG: start task:61b0c9ce-28eb-4c37-abd2-8ca06aa2557d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:61b0c9ce-28eb-4c37-abd2-8ca06aa2557d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 55772eba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 55772eba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 55772eba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4283 (Wed Sep 12 17:43:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4283 (Wed Sep 12 17:43:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c2db9ab", "local_conf_timestamp": 4283, "host-ts": 4283}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4275 (Wed Sep 12 17:43:06 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4275 (Wed Sep 12 17:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false} lago.ssh: DEBUG: start task:00896e68-46d0-433f-b3c1-dc141533f957:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:00896e68-46d0-433f-b3c1-dc141533f957:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 584d94c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 584d94c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 584d94c6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4283 (Wed Sep 12 17:43:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4283 (Wed Sep 12 17:43:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c2db9ab", "local_conf_timestamp": 4283, "host-ts": 4283}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4275 (Wed Sep 12 17:43:06 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4275 (Wed Sep 12 17:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false} lago.ssh: DEBUG: start task:a0067a07-37d5-438f-9816-a6f7f4ef82cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a0067a07-37d5-438f-9816-a6f7f4ef82cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5b2e424e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5b2e424e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5b2e424e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4293 (Wed Sep 12 17:43:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4293 (Wed Sep 12 17:43:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4405b336", "local_conf_timestamp": 4293, "host-ts": 4293}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4286 (Wed Sep 12 17:43:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4286 (Wed Sep 12 17:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false} lago.ssh: DEBUG: start task:37cb1c78-8c21-42dd-a5ce-b725e26d5bc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:37cb1c78-8c21-42dd-a5ce-b725e26d5bc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5c385eb8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5c385eb8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5c385eb8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4293 (Wed Sep 12 17:43:24 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4293 (Wed Sep 12 17:43:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4405b336", "local_conf_timestamp": 4293, "host-ts": 4293}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4286 (Wed Sep 12 17:43:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4286 (Wed Sep 12 17:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:4dfbd456-2c6b-43b5-bcc2-e48da17ca634:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4dfbd456-2c6b-43b5-bcc2-e48da17ca634:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 102500c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none lago.ssh: DEBUG: Command 102500c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for Engine Maintenance to reset... lago.ssh: DEBUG: start task:f4f0a020-fe3e-4573-a96b-1ca437b9d5e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f4f0a020-fe3e-4573-a96b-1ca437b9d5e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1276eb18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1276eb18 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1276eb18 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4597 (Wed Sep 12 17:48:28 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4597 (Wed Sep 12 17:48:28 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8fba211e", "local_conf_timestamp": 4597, "host-ts": 4597}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4597 (Wed Sep 12 17:48:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4597 (Wed Sep 12 17:48:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "afa0aca3", "local_conf_timestamp": 4597, "host-ts": 4597}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 130, in local_maintenance _wait_for_engine_maintenance(host, he_index, False) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 69, in _wait_for_engine_maintenance testlib.assert_true_within_long(lambda: _get_he_status(host) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:9d9e8dc5-3f92-44fa-acf7-71c866210045:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:9d9e8dc5-3f92-44fa-acf7-71c866210045:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 291925b2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 291925b2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 291925b2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Wed Sep 12 17:41:54 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4203 (Wed Sep 12 17:41:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "372f84c4", "local_conf_timestamp": 4203, "host-ts": 4203}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Wed Sep 12 17:41:56 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4206 (Wed Sep 12 17:41:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "80f9e1b9", "local_conf_timestamp": 4206, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:02edf3d0-3c34-4262-8b9e-3c8a78d0dc96:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:02edf3d0-3c34-4262-8b9e-3c8a78d0dc96:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2a581f8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 2a581f8c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:665c45d5-74a7-4e29-b68c-2044db74249f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:665c45d5-74a7-4e29-b68c-2044db74249f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2b76e72c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2b76e72c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2b76e72c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Wed Sep 12 17:42:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4214 (Wed Sep 12 17:42:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "752c5158", "local_conf_timestamp": 4214, "host-ts": 4213}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Wed Sep 12 17:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4215 (Wed Sep 12 17:42:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1db64f06", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9db09ee2-8a44-4206-a125-f263717738f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9db09ee2-8a44-4206-a125-f263717738f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 32d5a1b6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 32d5a1b6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 32d5a1b6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Wed Sep 12 17:42:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4223 (Wed Sep 12 17:42:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "77ff55d7", "local_conf_timestamp": 4223, "host-ts": 4223}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Wed Sep 12 17:42:05 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4215 (Wed Sep 12 17:42:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1db64f06", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:55195913-d8cb-4866-926e-042ae540d44c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:55195913-d8cb-4866-926e-042ae540d44c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 36822712 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 36822712 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 36822712 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Wed Sep 12 17:42:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4233 (Wed Sep 12 17:42:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34c7197b", "local_conf_timestamp": 4233, "host-ts": 4233}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Wed Sep 12 17:42:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4225 (Wed Sep 12 17:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d784fa39-da7d-4e2b-bc29-5c4ccfc1510f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d784fa39-da7d-4e2b-bc29-5c4ccfc1510f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3972006e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3972006e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3972006e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Wed Sep 12 17:42:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4233 (Wed Sep 12 17:42:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "34c7197b", "local_conf_timestamp": 4233, "host-ts": 4233}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Wed Sep 12 17:42:16 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4225 (Wed Sep 12 17:42:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "117d4e09", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:16d0a7de-b1f2-4116-91c3-80bd8592b430:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:16d0a7de-b1f2-4116-91c3-80bd8592b430:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3c902ffa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3c902ffa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3c902ffa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Wed Sep 12 17:42:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4243 (Wed Sep 12 17:42:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c8bcadb7", "local_conf_timestamp": 4243, "host-ts": 4243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Wed Sep 12 17:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4235 (Wed Sep 12 17:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "62068494", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:761b2a17-2dc8-4592-a8cd-5284fb7a8ea8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:761b2a17-2dc8-4592-a8cd-5284fb7a8ea8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3f968244 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3f968244 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3f968244 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Wed Sep 12 17:42:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4243 (Wed Sep 12 17:42:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c8bcadb7", "local_conf_timestamp": 4243, "host-ts": 4243}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Wed Sep 12 17:42:26 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4235 (Wed Sep 12 17:42:26 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "62068494", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f4cef96a-6372-4711-be92-4b3d0b68df80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f4cef96a-6372-4711-be92-4b3d0b68df80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 42b1d640 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 42b1d640 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 42b1d640 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Wed Sep 12 17:42:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4254 (Wed Sep 12 17:42:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f8eb06d6", "local_conf_timestamp": 4254, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Wed Sep 12 17:42:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Wed Sep 12 17:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e6df2835", "local_conf_timestamp": 4246, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:594d0f44-f3a9-422d-9360-7862752a3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:594d0f44-f3a9-422d-9360-7862752a3ebc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 45cc9518 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 45cc9518 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 45cc9518 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Wed Sep 12 17:42:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4254 (Wed Sep 12 17:42:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f8eb06d6", "local_conf_timestamp": 4254, "host-ts": 4253}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Wed Sep 12 17:42:36 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4246 (Wed Sep 12 17:42:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e6df2835", "local_conf_timestamp": 4246, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:008a1a58-ce1d-4bf8-b33a-b572228a1c4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:008a1a58-ce1d-4bf8-b33a-b572228a1c4d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 48c7eda8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 48c7eda8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 48c7eda8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Wed Sep 12 17:42:54 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4263 (Wed Sep 12 17:42:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4fabd672", "local_conf_timestamp": 4263, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Wed Sep 12 17:42:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Wed Sep 12 17:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f7c2e934-99e9-47fc-88bb-310b327fa881:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f7c2e934-99e9-47fc-88bb-310b327fa881:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4bf1af3c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4bf1af3c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4bf1af3c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Wed Sep 12 17:42:54 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4263 (Wed Sep 12 17:42:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4fabd672", "local_conf_timestamp": 4263, "host-ts": 4263}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Wed Sep 12 17:42:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Wed Sep 12 17:42:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7b90868a", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3b21f3e7-2780-4f8a-9bd1-4332137f0abe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3b21f3e7-2780-4f8a-9bd1-4332137f0abe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4f0b5d8a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4f0b5d8a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4f0b5d8a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Wed Sep 12 17:43:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4273 (Wed Sep 12 17:43:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e198e8b0", "local_conf_timestamp": 4273, "host-ts": 4273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4265 (Wed Sep 12 17:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4265 (Wed Sep 12 17:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c47f5f0b", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:57195e55-2edf-498b-9dd6-8f409872f39d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:57195e55-2edf-498b-9dd6-8f409872f39d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 52346bb4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 52346bb4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 52346bb4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4273 (Wed Sep 12 17:43:04 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4273 (Wed Sep 12 17:43:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e198e8b0", "local_conf_timestamp": 4273, "host-ts": 4273}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4265 (Wed Sep 12 17:42:56 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4265 (Wed Sep 12 17:42:56 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c47f5f0b", "local_conf_timestamp": 4265, "host-ts": 4265}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:61b0c9ce-28eb-4c37-abd2-8ca06aa2557d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:61b0c9ce-28eb-4c37-abd2-8ca06aa2557d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 55772eba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 55772eba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 55772eba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4283 (Wed Sep 12 17:43:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4283 (Wed Sep 12 17:43:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c2db9ab", "local_conf_timestamp": 4283, "host-ts": 4283}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4275 (Wed Sep 12 17:43:06 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4275 (Wed Sep 12 17:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:00896e68-46d0-433f-b3c1-dc141533f957:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:00896e68-46d0-433f-b3c1-dc141533f957:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 584d94c6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 584d94c6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 584d94c6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4283 (Wed Sep 12 17:43:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4283 (Wed Sep 12 17:43:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6c2db9ab", "local_conf_timestamp": 4283, "host-ts": 4283}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4275 (Wed Sep 12 17:43:06 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4275 (Wed Sep 12 17:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c039a00e", "local_conf_timestamp": 4275, "host-ts": 4275}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a0067a07-37d5-438f-9816-a6f7f4ef82cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a0067a07-37d5-438f-9816-a6f7f4ef82cc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5b2e424e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5b2e424e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5b2e424e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4293 (Wed Sep 12 17:43:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4293 (Wed Sep 12 17:43:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4405b336", "local_conf_timestamp": 4293, "host-ts": 4293}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4286 (Wed Sep 12 17:43:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4286 (Wed Sep 12 17:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:37cb1c78-8c21-42dd-a5ce-b725e26d5bc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:37cb1c78-8c21-42dd-a5ce-b725e26d5bc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5c385eb8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5c385eb8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5c385eb8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4293 (Wed Sep 12 17:43:24 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4293 (Wed Sep 12 17:43:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4405b336", "local_conf_timestamp": 4293, "host-ts": 4293}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4286 (Wed Sep 12 17:43:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4286 (Wed Sep 12 17:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "754fe743", "local_conf_timestamp": 4286, "host-ts": 4286}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:4dfbd456-2c6b-43b5-bcc2-e48da17ca634:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4dfbd456-2c6b-43b5-bcc2-e48da17ca634:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 102500c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none\nlago.ssh: DEBUG: Command 102500c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for Engine Maintenance to reset...\nlago.ssh: DEBUG: start task:f4f0a020-fe3e-4573-a96b-1ca437b9d5e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f4f0a020-fe3e-4573-a96b-1ca437b9d5e8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1276eb18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1276eb18 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1276eb18 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4597 (Wed Sep 12 17:48:28 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4597 (Wed Sep 12 17:48:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8fba211e", "local_conf_timestamp": 4597, "host-ts": 4597}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4597 (Wed Sep 12 17:48:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4597 (Wed Sep 12 17:48:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "afa0aca3", "local_conf_timestamp": 4597, "host-ts": 4597}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/307/ Build Number: 307 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... root: INFO: * Performing Deactivation... root: INFO: * Performing Activation... root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 79, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\nroot: INFO: * Performing Activation...\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/309/ Build Number: 309 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:a0ca46b7-3f9f-44e4-89b1-52f72aa44568:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:a0ca46b7-3f9f-44e4-89b1-52f72aa44568:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running ecd36f60 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ecd36f60 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command ecd36f60 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4174 (Fri Sep 14 01:38:40 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4174 (Fri Sep 14 01:38:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4172 (Fri Sep 14 01:38:38 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4172 (Fri Sep 14 01:38:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false} lago.ssh: DEBUG: start task:b7558a9a-f7d1-48c8-a744-fe29eade6e9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b7558a9a-f7d1-48c8-a744-fe29eade6e9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee0233e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command ee0233e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:2345c61a-4f8e-40d7-aaef-8e6d96209cde:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2345c61a-4f8e-40d7-aaef-8e6d96209cde:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ef255cf6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ef255cf6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ef255cf6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Fri Sep 14 01:38:50 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4185 (Fri Sep 14 01:38:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b51bae61", "local_conf_timestamp": 4185, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Fri Sep 14 01:38:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4183 (Fri Sep 14 01:38:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb9ed13d", "local_conf_timestamp": 4183, "host-ts": 4182}, "global_maintenance": false} lago.ssh: DEBUG: start task:e4d92dee-9502-43fe-b056-df000d690b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e4d92dee-9502-43fe-b056-df000d690b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f6f34d44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f6f34d44 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f6f34d44 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4194 (Fri Sep 14 01:39:00 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4194 (Fri Sep 14 01:39:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d1489e4", "local_conf_timestamp": 4194, "host-ts": 4194}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4192 (Fri Sep 14 01:38:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4192 (Fri Sep 14 01:38:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8df1eae", "local_conf_timestamp": 4192, "host-ts": 4192}, "global_maintenance": false} lago.ssh: DEBUG: start task:dcaec457-cd55-4412-941f-a7db1aba862f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dcaec457-cd55-4412-941f-a7db1aba862f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fafadc7c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fafadc7c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fafadc7c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4204 (Fri Sep 14 01:39:10 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4204 (Fri Sep 14 01:39:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dfdfec27", "local_conf_timestamp": 4204, "host-ts": 4204}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Fri Sep 14 01:39:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Fri Sep 14 01:39:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false} lago.ssh: DEBUG: start task:f1625c66-2cf3-40ec-9f90-b43da5393bac:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f1625c66-2cf3-40ec-9f90-b43da5393bac:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fe1d213a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fe1d213a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fe1d213a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4204 (Fri Sep 14 01:39:10 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4204 (Fri Sep 14 01:39:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dfdfec27", "local_conf_timestamp": 4204, "host-ts": 4204}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Fri Sep 14 01:39:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4202 (Fri Sep 14 01:39:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false} lago.ssh: DEBUG: start task:3c47ba3c-1d36-40fd-93cf-ee467d7b327a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3c47ba3c-1d36-40fd-93cf-ee467d7b327a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 014ec584 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 014ec584 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 014ec584 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4214 (Fri Sep 14 01:39:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4214 (Fri Sep 14 01:39:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Fri Sep 14 01:39:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4212 (Fri Sep 14 01:39:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false} lago.ssh: DEBUG: start task:69c57dbd-7e8e-42ca-8e66-28469d79f1d9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:69c57dbd-7e8e-42ca-8e66-28469d79f1d9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 05299616 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 05299616 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 05299616 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4214 (Fri Sep 14 01:39:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4214 (Fri Sep 14 01:39:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4212 (Fri Sep 14 01:39:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4212 (Fri Sep 14 01:39:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false} lago.ssh: DEBUG: start task:a13bc8b4-2ef1-480a-9c0c-615a568d0c29:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a13bc8b4-2ef1-480a-9c0c-615a568d0c29:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0852ddb6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0852ddb6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0852ddb6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Fri Sep 14 01:39:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4224 (Fri Sep 14 01:39:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "58c897e2", "local_conf_timestamp": 4224, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Fri Sep 14 01:39:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4223 (Fri Sep 14 01:39:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "76aa3417", "local_conf_timestamp": 4223, "host-ts": 4222}, "global_maintenance": false} lago.ssh: DEBUG: start task:732e8041-cc56-4ade-8977-bed0e7292a8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:732e8041-cc56-4ade-8977-bed0e7292a8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0b5405da on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0b5405da on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0b5405da on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4224 (Fri Sep 14 01:39:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4224 (Fri Sep 14 01:39:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "58c897e2", "local_conf_timestamp": 4224, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4222 (Fri Sep 14 01:39:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4223 (Fri Sep 14 01:39:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "76aa3417", "local_conf_timestamp": 4223, "host-ts": 4222}, "global_maintenance": false} lago.ssh: DEBUG: start task:551f813f-9775-4707-9c95-3b3fdc76ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:551f813f-9775-4707-9c95-3b3fdc76ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0e6042ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0e6042ca on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0e6042ca on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Fri Sep 14 01:39:40 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4234 (Fri Sep 14 01:39:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Fri Sep 14 01:39:38 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4232 (Fri Sep 14 01:39:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b6cf230a", "local_conf_timestamp": 4232, "host-ts": 4232}, "global_maintenance": false} lago.ssh: DEBUG: start task:3f99022d-cce8-4d26-bd70-ebf230f9313f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3f99022d-cce8-4d26-bd70-ebf230f9313f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 117ddefe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 117ddefe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 117ddefe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4234 (Fri Sep 14 01:39:40 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4234 (Fri Sep 14 01:39:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4232 (Fri Sep 14 01:39:38 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4232 (Fri Sep 14 01:39:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b6cf230a", "local_conf_timestamp": 4232, "host-ts": 4232}, "global_maintenance": false} lago.ssh: DEBUG: start task:57788e2c-782f-4a59-aa25-ec03dd7b2d54:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:57788e2c-782f-4a59-aa25-ec03dd7b2d54:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 14ef0ed2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 14ef0ed2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 14ef0ed2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4244 (Fri Sep 14 01:39:50 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4245 (Fri Sep 14 01:39:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d872d7a", "local_conf_timestamp": 4245, "host-ts": 4244}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4242 (Fri Sep 14 01:39:48 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4242 (Fri Sep 14 01:39:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ab1fde11", "local_conf_timestamp": 4242, "host-ts": 4242}, "global_maintenance": false} lago.ssh: DEBUG: start task:d2af6454-aecb-496d-81c8-064660f951fe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d2af6454-aecb-496d-81c8-064660f951fe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 17e7cb42 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 17e7cb42 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 17e7cb42 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4254 (Fri Sep 14 01:40:00 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4254 (Fri Sep 14 01:40:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b3232e", "local_conf_timestamp": 4254, "host-ts": 4254}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4252 (Fri Sep 14 01:39:58 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4252 (Fri Sep 14 01:39:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "af592114", "local_conf_timestamp": 4252, "host-ts": 4252}, "global_maintenance": false} lago.ssh: DEBUG: start task:a9d705cc-42fd-4d84-bc16-93a3f145c660:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a9d705cc-42fd-4d84-bc16-93a3f145c660:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1b0b6de2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1b0b6de2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1b0b6de2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4254 (Fri Sep 14 01:40:00 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4254 (Fri Sep 14 01:40:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b3232e", "local_conf_timestamp": 4254, "host-ts": 4254}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4252 (Fri Sep 14 01:39:58 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4252 (Fri Sep 14 01:39:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "af592114", "local_conf_timestamp": 4252, "host-ts": 4252}, "global_maintenance": false} lago.ssh: DEBUG: start task:6d4f079f-b392-40c1-a068-9056bfaf955a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6d4f079f-b392-40c1-a068-9056bfaf955a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1de93c88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1de93c88 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1de93c88 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4264 (Fri Sep 14 01:40:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4264 (Fri Sep 14 01:40:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c0fc915", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Fri Sep 14 01:40:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4263 (Fri Sep 14 01:40:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9de949ea", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false} lago.ssh: DEBUG: start task:85136506-a89f-4f85-9ffa-54c29a6fee8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:85136506-a89f-4f85-9ffa-54c29a6fee8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1efada64 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1efada64 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1efada64 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4264 (Fri Sep 14 01:40:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4264 (Fri Sep 14 01:40:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c0fc915", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Fri Sep 14 01:40:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4263 (Fri Sep 14 01:40:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9de949ea", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:aecac330-251e-4e6c-baf9-4087b3b7da05:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aecac330-251e-4e6c-baf9-4087b3b7da05:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d2f2fb0a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none lago.ssh: DEBUG: Command d2f2fb0a on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for Engine Maintenance to reset... lago.ssh: DEBUG: start task:5ba79776-78c8-4fb3-a53a-9890d98a6762:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5ba79776-78c8-4fb3-a53a-9890d98a6762:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d5411f86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d5411f86 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d5411f86 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4565 (Fri Sep 14 01:45:12 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4566 (Fri Sep 14 01:45:12 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "e3d8e834", "local_conf_timestamp": 4566, "host-ts": 4565}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4565 (Fri Sep 14 01:45:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4565 (Fri Sep 14 01:45:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3b629e7d", "local_conf_timestamp": 4565, "host-ts": 4565}, "global_maintenance": false} lago.ssh: DEBUG: start task:b38c51f9-aaa4-404a-bce3-803990d02096:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b38c51f9-aaa4-404a-bce3-803990d02096:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dc50815e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dc50815e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dc50815e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4576 (Fri Sep 14 01:45:22 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4576 (Fri Sep 14 01:45:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "facd8470", "local_conf_timestamp": 4576, "host-ts": 4576}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4575 (Fri Sep 14 01:45:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4575 (Fri Sep 14 01:45:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e705de42", "local_conf_timestamp": 4575, "host-ts": 4575}, "global_maintenance": false} lago.ssh: DEBUG: start task:1e17320a-fa99-4dcf-a87a-a8a98d9e0fa1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1e17320a-fa99-4dcf-a87a-a8a98d9e0fa1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dd55beca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dd55beca on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dd55beca on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4576 (Fri Sep 14 01:45:22 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4576 (Fri Sep 14 01:45:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "facd8470", "local_conf_timestamp": 4576, "host-ts": 4576}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4575 (Fri Sep 14 01:45:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4575 (Fri Sep 14 01:45:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e705de42", "local_conf_timestamp": 4575, "host-ts": 4575}, "global_maintenance": false} cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 130, in local_maintenance _wait_for_engine_maintenance(host, he_index, False) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 73, in _wait_for_engine_maintenance lambda: _is_state_maintenance(host, "LocalMaintenance") is False File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 248, in assert_equals_within time.sleep(initial_wait) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:a0ca46b7-3f9f-44e4-89b1-52f72aa44568:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:a0ca46b7-3f9f-44e4-89b1-52f72aa44568:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running ecd36f60 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ecd36f60 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command ecd36f60 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4174 (Fri Sep 14 01:38:40 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4174 (Fri Sep 14 01:38:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4172 (Fri Sep 14 01:38:38 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4172 (Fri Sep 14 01:38:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f37ee498", "local_conf_timestamp": 4172, "host-ts": 4172}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b7558a9a-f7d1-48c8-a744-fe29eade6e9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b7558a9a-f7d1-48c8-a744-fe29eade6e9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee0233e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command ee0233e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:2345c61a-4f8e-40d7-aaef-8e6d96209cde:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2345c61a-4f8e-40d7-aaef-8e6d96209cde:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ef255cf6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ef255cf6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ef255cf6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Fri Sep 14 01:38:50 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4185 (Fri Sep 14 01:38:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b51bae61", "local_conf_timestamp": 4185, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Fri Sep 14 01:38:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4183 (Fri Sep 14 01:38:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bb9ed13d", "local_conf_timestamp": 4183, "host-ts": 4182}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e4d92dee-9502-43fe-b056-df000d690b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e4d92dee-9502-43fe-b056-df000d690b93:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f6f34d44 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f6f34d44 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f6f34d44 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4194 (Fri Sep 14 01:39:00 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4194 (Fri Sep 14 01:39:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d1489e4", "local_conf_timestamp": 4194, "host-ts": 4194}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4192 (Fri Sep 14 01:38:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4192 (Fri Sep 14 01:38:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c8df1eae", "local_conf_timestamp": 4192, "host-ts": 4192}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dcaec457-cd55-4412-941f-a7db1aba862f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dcaec457-cd55-4412-941f-a7db1aba862f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fafadc7c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fafadc7c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fafadc7c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4204 (Fri Sep 14 01:39:10 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4204 (Fri Sep 14 01:39:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dfdfec27", "local_conf_timestamp": 4204, "host-ts": 4204}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Fri Sep 14 01:39:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Fri Sep 14 01:39:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f1625c66-2cf3-40ec-9f90-b43da5393bac:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f1625c66-2cf3-40ec-9f90-b43da5393bac:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fe1d213a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fe1d213a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fe1d213a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4204 (Fri Sep 14 01:39:10 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4204 (Fri Sep 14 01:39:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "dfdfec27", "local_conf_timestamp": 4204, "host-ts": 4204}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Fri Sep 14 01:39:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4202 (Fri Sep 14 01:39:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0920fa8b", "local_conf_timestamp": 4202, "host-ts": 4202}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3c47ba3c-1d36-40fd-93cf-ee467d7b327a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3c47ba3c-1d36-40fd-93cf-ee467d7b327a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 014ec584 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 014ec584 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 014ec584 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4214 (Fri Sep 14 01:39:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4214 (Fri Sep 14 01:39:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Fri Sep 14 01:39:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4212 (Fri Sep 14 01:39:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:69c57dbd-7e8e-42ca-8e66-28469d79f1d9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:69c57dbd-7e8e-42ca-8e66-28469d79f1d9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 05299616 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 05299616 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 05299616 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4214 (Fri Sep 14 01:39:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4214 (Fri Sep 14 01:39:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71ecd2e5", "local_conf_timestamp": 4214, "host-ts": 4214}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4212 (Fri Sep 14 01:39:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4212 (Fri Sep 14 01:39:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "0d66058e", "local_conf_timestamp": 4212, "host-ts": 4212}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a13bc8b4-2ef1-480a-9c0c-615a568d0c29:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a13bc8b4-2ef1-480a-9c0c-615a568d0c29:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0852ddb6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0852ddb6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0852ddb6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Fri Sep 14 01:39:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4224 (Fri Sep 14 01:39:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "58c897e2", "local_conf_timestamp": 4224, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Fri Sep 14 01:39:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4223 (Fri Sep 14 01:39:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "76aa3417", "local_conf_timestamp": 4223, "host-ts": 4222}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:732e8041-cc56-4ade-8977-bed0e7292a8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:732e8041-cc56-4ade-8977-bed0e7292a8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0b5405da on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0b5405da on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0b5405da on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4224 (Fri Sep 14 01:39:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4224 (Fri Sep 14 01:39:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "58c897e2", "local_conf_timestamp": 4224, "host-ts": 4224}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4222 (Fri Sep 14 01:39:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4223 (Fri Sep 14 01:39:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "76aa3417", "local_conf_timestamp": 4223, "host-ts": 4222}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:551f813f-9775-4707-9c95-3b3fdc76ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:551f813f-9775-4707-9c95-3b3fdc76ba8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0e6042ca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0e6042ca on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0e6042ca on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Fri Sep 14 01:39:40 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4234 (Fri Sep 14 01:39:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Fri Sep 14 01:39:38 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4232 (Fri Sep 14 01:39:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b6cf230a", "local_conf_timestamp": 4232, "host-ts": 4232}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3f99022d-cce8-4d26-bd70-ebf230f9313f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3f99022d-cce8-4d26-bd70-ebf230f9313f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 117ddefe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 117ddefe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 117ddefe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4234 (Fri Sep 14 01:39:40 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4234 (Fri Sep 14 01:39:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f6fba920", "local_conf_timestamp": 4234, "host-ts": 4234}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4232 (Fri Sep 14 01:39:38 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4232 (Fri Sep 14 01:39:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b6cf230a", "local_conf_timestamp": 4232, "host-ts": 4232}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:57788e2c-782f-4a59-aa25-ec03dd7b2d54:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:57788e2c-782f-4a59-aa25-ec03dd7b2d54:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 14ef0ed2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 14ef0ed2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 14ef0ed2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4244 (Fri Sep 14 01:39:50 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4245 (Fri Sep 14 01:39:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d872d7a", "local_conf_timestamp": 4245, "host-ts": 4244}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4242 (Fri Sep 14 01:39:48 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4242 (Fri Sep 14 01:39:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ab1fde11", "local_conf_timestamp": 4242, "host-ts": 4242}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d2af6454-aecb-496d-81c8-064660f951fe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d2af6454-aecb-496d-81c8-064660f951fe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 17e7cb42 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 17e7cb42 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 17e7cb42 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4254 (Fri Sep 14 01:40:00 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4254 (Fri Sep 14 01:40:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b3232e", "local_conf_timestamp": 4254, "host-ts": 4254}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4252 (Fri Sep 14 01:39:58 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4252 (Fri Sep 14 01:39:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "af592114", "local_conf_timestamp": 4252, "host-ts": 4252}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a9d705cc-42fd-4d84-bc16-93a3f145c660:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a9d705cc-42fd-4d84-bc16-93a3f145c660:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1b0b6de2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1b0b6de2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1b0b6de2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4254 (Fri Sep 14 01:40:00 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4254 (Fri Sep 14 01:40:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b3232e", "local_conf_timestamp": 4254, "host-ts": 4254}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4252 (Fri Sep 14 01:39:58 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4252 (Fri Sep 14 01:39:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "af592114", "local_conf_timestamp": 4252, "host-ts": 4252}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6d4f079f-b392-40c1-a068-9056bfaf955a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6d4f079f-b392-40c1-a068-9056bfaf955a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1de93c88 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1de93c88 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1de93c88 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4264 (Fri Sep 14 01:40:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4264 (Fri Sep 14 01:40:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c0fc915", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Fri Sep 14 01:40:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4263 (Fri Sep 14 01:40:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9de949ea", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:85136506-a89f-4f85-9ffa-54c29a6fee8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:85136506-a89f-4f85-9ffa-54c29a6fee8d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1efada64 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1efada64 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1efada64 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4264 (Fri Sep 14 01:40:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4264 (Fri Sep 14 01:40:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c0fc915", "local_conf_timestamp": 4264, "host-ts": 4264}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Fri Sep 14 01:40:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4263 (Fri Sep 14 01:40:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9de949ea", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:aecac330-251e-4e6c-baf9-4087b3b7da05:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aecac330-251e-4e6c-baf9-4087b3b7da05:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d2f2fb0a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=none\nlago.ssh: DEBUG: Command d2f2fb0a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for Engine Maintenance to reset...\nlago.ssh: DEBUG: start task:5ba79776-78c8-4fb3-a53a-9890d98a6762:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5ba79776-78c8-4fb3-a53a-9890d98a6762:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d5411f86 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d5411f86 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d5411f86 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4565 (Fri Sep 14 01:45:12 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4566 (Fri Sep 14 01:45:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "e3d8e834", "local_conf_timestamp": 4566, "host-ts": 4565}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4565 (Fri Sep 14 01:45:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4565 (Fri Sep 14 01:45:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3b629e7d", "local_conf_timestamp": 4565, "host-ts": 4565}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b38c51f9-aaa4-404a-bce3-803990d02096:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b38c51f9-aaa4-404a-bce3-803990d02096:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dc50815e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dc50815e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dc50815e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4576 (Fri Sep 14 01:45:22 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4576 (Fri Sep 14 01:45:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "facd8470", "local_conf_timestamp": 4576, "host-ts": 4576}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4575 (Fri Sep 14 01:45:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4575 (Fri Sep 14 01:45:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e705de42", "local_conf_timestamp": 4575, "host-ts": 4575}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1e17320a-fa99-4dcf-a87a-a8a98d9e0fa1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1e17320a-fa99-4dcf-a87a-a8a98d9e0fa1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dd55beca on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dd55beca on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dd55beca on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4576 (Fri Sep 14 01:45:22 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4576 (Fri Sep 14 01:45:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "facd8470", "local_conf_timestamp": 4576, "host-ts": 4576}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4575 (Fri Sep 14 01:45:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4575 (Fri Sep 14 01:45:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e705de42", "local_conf_timestamp": 4575, "host-ts": 4575}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/311/ Build Number: 311 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/313/ Build Number: 313 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:88d4e3c2-f6d2-49f9-a59b-48a6f6b2d9d5:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:88d4e3c2-f6d2-49f9-a59b-48a6f6b2d9d5:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running c6503e46 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c6503e46 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command c6503e46 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Sat Sep 15 09:43:11 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4177 (Sat Sep 15 09:43:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7935c07", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sat Sep 15 09:43:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4175 (Sat Sep 15 09:43:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94939b88", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:afc8c9ea-1b2a-4b03-8933-57da302bb866:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:afc8c9ea-1b2a-4b03-8933-57da302bb866:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c76b244e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command c76b244e on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:9d8949b3-4891-4fb2-80c3-e6ebdc3feb22:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9d8949b3-4891-4fb2-80c3-e6ebdc3feb22:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c87e2df4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c87e2df4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c87e2df4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4187 (Sat Sep 15 09:43:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Sat Sep 15 09:43:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "796bc279", "local_conf_timestamp": 4188, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sat Sep 15 09:43:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Sat Sep 15 09:43:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ab749ebb", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:49993f86-bdcb-4b02-891d-5461718a9791:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:49993f86-bdcb-4b02-891d-5461718a9791:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d043fd66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d043fd66 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d043fd66 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Sat Sep 15 09:43:31 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Sat Sep 15 09:43:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sat Sep 15 09:43:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Sat Sep 15 09:43:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4106059c", "local_conf_timestamp": 4196, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:e52f16c2-ee5b-475b-8a8d-1df0de49f588:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e52f16c2-ee5b-475b-8a8d-1df0de49f588:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d32d5266 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d32d5266 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d32d5266 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Sat Sep 15 09:43:31 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Sat Sep 15 09:43:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sat Sep 15 09:43:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Sat Sep 15 09:43:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4106059c", "local_conf_timestamp": 4196, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:2b7b693e-63b0-42bd-b928-3ffbd8868456:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2b7b693e-63b0-42bd-b928-3ffbd8868456:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d62c38d8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d62c38d8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d62c38d8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Sat Sep 15 09:43:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Sat Sep 15 09:43:42 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sat Sep 15 09:43:38 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Sat Sep 15 09:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:427e8e0d-9959-48c5-b95c-cdc51de02591:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:427e8e0d-9959-48c5-b95c-cdc51de02591:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running d94e7a4e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command d94e7a4e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command d94e7a4e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Sat Sep 15 09:43:42 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Sat Sep 15 09:43:42 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sat Sep 15 09:43:38 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Sat Sep 15 09:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:9cb311b0-4a62-4cfe-9930-5865e6868478:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9cb311b0-4a62-4cfe-9930-5865e6868478:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dc36dc2e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dc36dc2e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command dc36dc2e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Sat Sep 15 09:43:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Sat Sep 15 09:43:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sat Sep 15 09:43:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4215 (Sat Sep 15 09:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:0204e396-9bea-4d71-a07d-362a810e0e69:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0204e396-9bea-4d71-a07d-362a810e0e69:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running df27a742 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command df27a742 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command df27a742 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Sat Sep 15 09:43:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Sat Sep 15 09:43:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sat Sep 15 09:43:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4215 (Sat Sep 15 09:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:26d8a6cd-2470-4244-b57d-38022681305a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:26d8a6cd-2470-4244-b57d-38022681305a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e2168ebe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e2168ebe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e2168ebe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sat Sep 15 09:44:01 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sat Sep 15 09:44:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sat Sep 15 09:43:59 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4225 (Sat Sep 15 09:43:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:8b76ed69-8a8f-464f-b06e-d91686b82e51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8b76ed69-8a8f-464f-b06e-d91686b82e51:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e5108a84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e5108a84 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e5108a84 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sat Sep 15 09:44:01 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sat Sep 15 09:44:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sat Sep 15 09:43:59 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4225 (Sat Sep 15 09:43:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:0c6ea76f-3926-4237-9cd5-b7991e74b934:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0c6ea76f-3926-4237-9cd5-b7991e74b934:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e80c24dc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e80c24dc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e80c24dc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Sat Sep 15 09:44:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Sat Sep 15 09:44:12 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sat Sep 15 09:44:09 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4235 (Sat Sep 15 09:44:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:fa5b4ef7-2703-45d3-bf31-c99044b6c436:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fa5b4ef7-2703-45d3-bf31-c99044b6c436:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running eb1db654 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command eb1db654 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command eb1db654 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Sat Sep 15 09:44:12 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Sat Sep 15 09:44:12 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sat Sep 15 09:44:09 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4235 (Sat Sep 15 09:44:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:08febe18-1cea-4780-9159-3c3e8fb0e99b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:08febe18-1cea-4780-9159-3c3e8fb0e99b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ee93bfb8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ee93bfb8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ee93bfb8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Sat Sep 15 09:44:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4248 (Sat Sep 15 09:44:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Sat Sep 15 09:44:18 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4245 (Sat Sep 15 09:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:cf86ee66-7270-4602-9fe8-2e67d769e0db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:cf86ee66-7270-4602-9fe8-2e67d769e0db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f1714c78 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f1714c78 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f1714c78 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Sat Sep 15 09:44:21 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4248 (Sat Sep 15 09:44:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4245 (Sat Sep 15 09:44:18 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4245 (Sat Sep 15 09:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false} lago.ssh: DEBUG: start task:12b565c8-bc33-4f24-b09c-510894c8b753:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:12b565c8-bc33-4f24-b09c-510894c8b753:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f4500b28 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f4500b28 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f4500b28 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Sat Sep 15 09:44:32 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4258 (Sat Sep 15 09:44:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3fdd1177", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Sat Sep 15 09:44:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Sat Sep 15 09:44:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} lago.ssh: DEBUG: start task:dbb5ac3f-1adb-4601-9673-d6497747acea:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dbb5ac3f-1adb-4601-9673-d6497747acea:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f55fea6a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f55fea6a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f55fea6a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Sat Sep 15 09:44:32 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4258 (Sat Sep 15 09:44:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3fdd1177", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4255 (Sat Sep 15 09:44:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4255 (Sat Sep 15 09:44:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:88d4e3c2-f6d2-49f9-a59b-48a6f6b2d9d5:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:88d4e3c2-f6d2-49f9-a59b-48a6f6b2d9d5:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running c6503e46 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c6503e46 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command c6503e46 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Sat Sep 15 09:43:11 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4177 (Sat Sep 15 09:43:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e7935c07", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sat Sep 15 09:43:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4175 (Sat Sep 15 09:43:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94939b88", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:afc8c9ea-1b2a-4b03-8933-57da302bb866:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:afc8c9ea-1b2a-4b03-8933-57da302bb866:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c76b244e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command c76b244e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:9d8949b3-4891-4fb2-80c3-e6ebdc3feb22:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9d8949b3-4891-4fb2-80c3-e6ebdc3feb22:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c87e2df4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c87e2df4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c87e2df4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4187 (Sat Sep 15 09:43:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Sat Sep 15 09:43:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "796bc279", "local_conf_timestamp": 4188, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sat Sep 15 09:43:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Sat Sep 15 09:43:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ab749ebb", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:49993f86-bdcb-4b02-891d-5461718a9791:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:49993f86-bdcb-4b02-891d-5461718a9791:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d043fd66 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d043fd66 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d043fd66 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Sat Sep 15 09:43:31 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Sat Sep 15 09:43:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sat Sep 15 09:43:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Sat Sep 15 09:43:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4106059c", "local_conf_timestamp": 4196, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e52f16c2-ee5b-475b-8a8d-1df0de49f588:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e52f16c2-ee5b-475b-8a8d-1df0de49f588:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d32d5266 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d32d5266 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d32d5266 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Sat Sep 15 09:43:31 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Sat Sep 15 09:43:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sat Sep 15 09:43:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Sat Sep 15 09:43:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4106059c", "local_conf_timestamp": 4196, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2b7b693e-63b0-42bd-b928-3ffbd8868456:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2b7b693e-63b0-42bd-b928-3ffbd8868456:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d62c38d8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d62c38d8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d62c38d8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Sat Sep 15 09:43:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Sat Sep 15 09:43:42 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sat Sep 15 09:43:38 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Sat Sep 15 09:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:427e8e0d-9959-48c5-b95c-cdc51de02591:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:427e8e0d-9959-48c5-b95c-cdc51de02591:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running d94e7a4e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command d94e7a4e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command d94e7a4e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Sat Sep 15 09:43:42 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Sat Sep 15 09:43:42 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sat Sep 15 09:43:38 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Sat Sep 15 09:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:9cb311b0-4a62-4cfe-9930-5865e6868478:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9cb311b0-4a62-4cfe-9930-5865e6868478:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dc36dc2e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dc36dc2e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command dc36dc2e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Sat Sep 15 09:43:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Sat Sep 15 09:43:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sat Sep 15 09:43:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4215 (Sat Sep 15 09:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0204e396-9bea-4d71-a07d-362a810e0e69:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0204e396-9bea-4d71-a07d-362a810e0e69:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running df27a742 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command df27a742 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command df27a742 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Sat Sep 15 09:43:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Sat Sep 15 09:43:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sat Sep 15 09:43:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4215 (Sat Sep 15 09:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6a8b7a9e", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:26d8a6cd-2470-4244-b57d-38022681305a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:26d8a6cd-2470-4244-b57d-38022681305a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e2168ebe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e2168ebe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e2168ebe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sat Sep 15 09:44:01 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sat Sep 15 09:44:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sat Sep 15 09:43:59 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4225 (Sat Sep 15 09:43:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8b76ed69-8a8f-464f-b06e-d91686b82e51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8b76ed69-8a8f-464f-b06e-d91686b82e51:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e5108a84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e5108a84 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e5108a84 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sat Sep 15 09:44:01 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sat Sep 15 09:44:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sat Sep 15 09:43:59 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4225 (Sat Sep 15 09:43:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0c6ea76f-3926-4237-9cd5-b7991e74b934:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0c6ea76f-3926-4237-9cd5-b7991e74b934:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e80c24dc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e80c24dc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e80c24dc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Sat Sep 15 09:44:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Sat Sep 15 09:44:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sat Sep 15 09:44:09 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4235 (Sat Sep 15 09:44:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fa5b4ef7-2703-45d3-bf31-c99044b6c436:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fa5b4ef7-2703-45d3-bf31-c99044b6c436:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running eb1db654 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command eb1db654 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command eb1db654 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Sat Sep 15 09:44:12 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Sat Sep 15 09:44:12 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b62bd846", "local_conf_timestamp": 4238, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sat Sep 15 09:44:09 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4235 (Sat Sep 15 09:44:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d1225c1a", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:08febe18-1cea-4780-9159-3c3e8fb0e99b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:08febe18-1cea-4780-9159-3c3e8fb0e99b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ee93bfb8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ee93bfb8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ee93bfb8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Sat Sep 15 09:44:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4248 (Sat Sep 15 09:44:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Sat Sep 15 09:44:18 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4245 (Sat Sep 15 09:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:cf86ee66-7270-4602-9fe8-2e67d769e0db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cf86ee66-7270-4602-9fe8-2e67d769e0db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f1714c78 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f1714c78 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f1714c78 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Sat Sep 15 09:44:21 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4248 (Sat Sep 15 09:44:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a506c8a", "local_conf_timestamp": 4248, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4245 (Sat Sep 15 09:44:18 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4245 (Sat Sep 15 09:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ccf2a101", "local_conf_timestamp": 4245, "host-ts": 4245}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:12b565c8-bc33-4f24-b09c-510894c8b753:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:12b565c8-bc33-4f24-b09c-510894c8b753:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f4500b28 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f4500b28 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f4500b28 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Sat Sep 15 09:44:32 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4258 (Sat Sep 15 09:44:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3fdd1177", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Sat Sep 15 09:44:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Sat Sep 15 09:44:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dbb5ac3f-1adb-4601-9673-d6497747acea:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dbb5ac3f-1adb-4601-9673-d6497747acea:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f55fea6a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f55fea6a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f55fea6a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Sat Sep 15 09:44:32 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4258 (Sat Sep 15 09:44:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "3fdd1177", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4255 (Sat Sep 15 09:44:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4255 (Sat Sep 15 09:44:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "337fa38c", "local_conf_timestamp": 4255, "host-ts": 4255}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/315/ Build Number: 315 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:57484a71-e9e6-4eca-a1ba-d4970477aaf4:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:57484a71-e9e6-4eca-a1ba-d4970477aaf4:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running dbeb02d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command dbeb02d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command dbeb02d2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4148 (Sun Sep 16 01:42:51 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4148 (Sun Sep 16 01:42:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7270d4f8", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4155 (Sun Sep 16 01:42:57 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4155 (Sun Sep 16 01:42:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c1e6582", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false} lago.ssh: DEBUG: start task:aa0e5b29-91c5-40a8-b074-28476460cb1b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aa0e5b29-91c5-40a8-b074-28476460cb1b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running dd0795cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command dd0795cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:922e35b9-25f1-4fd5-8083-0da21011ee6d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:922e35b9-25f1-4fd5-8083-0da21011ee6d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running de1f7cf4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command de1f7cf4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command de1f7cf4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4168 (Sun Sep 16 01:43:11 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4168 (Sun Sep 16 01:43:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32a005eb", "local_conf_timestamp": 4168, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Sun Sep 16 01:43:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4165 (Sun Sep 16 01:43:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "90d5648d", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false} lago.ssh: DEBUG: start task:efebb507-eed1-402c-900c-d8b991d64c98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:efebb507-eed1-402c-900c-d8b991d64c98:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e54f9eb4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e54f9eb4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e54f9eb4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Sun Sep 16 01:43:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4178 (Sun Sep 16 01:43:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sun Sep 16 01:43:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4175 (Sun Sep 16 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94939b88", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:3c4221fd-f368-4daa-9fd6-c7d9c62fe6e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3c4221fd-f368-4daa-9fd6-c7d9c62fe6e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running e91662f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command e91662f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command e91662f8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Sun Sep 16 01:43:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4178 (Sun Sep 16 01:43:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Sun Sep 16 01:43:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4175 (Sun Sep 16 01:43:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94939b88", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:fb8f553e-88a1-42b7-92fc-fea7ff17a32e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fb8f553e-88a1-42b7-92fc-fea7ff17a32e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ec062778 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ec062778 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ec062778 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Sun Sep 16 01:43:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Sun Sep 16 01:43:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Sep 16 01:43:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Sun Sep 16 01:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc49ab23", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:2d875d5d-1c8f-4900-a691-da5036cd0cd0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2d875d5d-1c8f-4900-a691-da5036cd0cd0:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ef0b27e8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ef0b27e8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ef0b27e8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Sun Sep 16 01:43:30 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4188 (Sun Sep 16 01:43:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Sun Sep 16 01:43:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Sun Sep 16 01:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc49ab23", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:d4dc8859-9e86-48d7-84c4-911339239213:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d4dc8859-9e86-48d7-84c4-911339239213:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f1fedfa8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f1fedfa8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f1fedfa8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Sun Sep 16 01:43:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Sun Sep 16 01:43:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Sep 16 01:43:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4195 (Sun Sep 16 01:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d80f5426", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:7dee7209-2ea3-473b-b364-cb7825e74580:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7dee7209-2ea3-473b-b364-cb7825e74580:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f4e7ed72 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f4e7ed72 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f4e7ed72 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Sun Sep 16 01:43:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Sun Sep 16 01:43:41 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4195 (Sun Sep 16 01:43:37 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4195 (Sun Sep 16 01:43:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d80f5426", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false} lago.ssh: DEBUG: start task:c46089f7-dbd0-4907-b521-e7a04d8c3779:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c46089f7-dbd0-4907-b521-e7a04d8c3779:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f7d2dd4e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command f7d2dd4e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command f7d2dd4e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Sun Sep 16 01:43:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Sun Sep 16 01:43:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 16 01:43:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Sun Sep 16 01:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:4b101b93-85db-4a08-b611-266c770f9b8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4b101b93-85db-4a08-b611-266c770f9b8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running faf06460 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command faf06460 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command faf06460 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Sun Sep 16 01:43:51 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Sun Sep 16 01:43:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Sun Sep 16 01:43:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4205 (Sun Sep 16 01:43:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false} lago.ssh: DEBUG: start task:f9afa89c-bccb-4b59-b3ea-532ac669fdc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f9afa89c-bccb-4b59-b3ea-532ac669fdc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running fe38ee4e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command fe38ee4e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command fe38ee4e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Sun Sep 16 01:44:00 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Sun Sep 16 01:44:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Sun Sep 16 01:43:57 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4215 (Sun Sep 16 01:43:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9afa210", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false} lago.ssh: DEBUG: start task:78fe242c-6aa9-4edb-9536-bb7ae9e479ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:78fe242c-6aa9-4edb-9536-bb7ae9e479ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 01615fb6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 01615fb6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 01615fb6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sun Sep 16 01:44:11 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sun Sep 16 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 16 01:44:07 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 16 01:44:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:762a8160-7a78-4f11-9480-671b38b5bf7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:762a8160-7a78-4f11-9480-671b38b5bf7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0503352c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0503352c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0503352c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sun Sep 16 01:44:11 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sun Sep 16 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Sun Sep 16 01:44:07 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4225 (Sun Sep 16 01:44:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false} lago.ssh: DEBUG: start task:ff044350-065e-485f-a443-a37aa4a70784:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ff044350-065e-485f-a443-a37aa4a70784:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 07dad32c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 07dad32c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 07dad32c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Sun Sep 16 01:44:11 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4228 (Sun Sep 16 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 16 01:44:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4235 (Sun Sep 16 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "97bf29cd", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:3f1e3607-915e-43f2-b21c-44cf9ff7ca22:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3f1e3607-915e-43f2-b21c-44cf9ff7ca22:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0aaf70ee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0aaf70ee on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0aaf70ee on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Sun Sep 16 01:44:21 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4239 (Sun Sep 16 01:44:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "46234a3f", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 16 01:44:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4235 (Sun Sep 16 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "97bf29cd", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} lago.ssh: DEBUG: start task:ef7bb398-b949-401a-a366-921d0ea537b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ef7bb398-b949-401a-a366-921d0ea537b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 0bc370c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 0bc370c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 0bc370c0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Sun Sep 16 01:44:21 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4239 (Sun Sep 16 01:44:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "46234a3f", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Sun Sep 16 01:44:17 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4235 (Sun Sep 16 01:44:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "97bf29cd", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:57484a71-e9e6-4eca-a1ba-d4970477aaf4:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:57484a71-e9e6-4eca-a1ba-d4970477aaf4:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running dbeb02d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command dbeb02d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command dbeb02d2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4148 (Sun Sep 16 01:42:51 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4148 (Sun Sep 16 01:42:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7270d4f8", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4155 (Sun Sep 16 01:42:57 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4155 (Sun Sep 16 01:42:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c1e6582", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aa0e5b29-91c5-40a8-b074-28476460cb1b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aa0e5b29-91c5-40a8-b074-28476460cb1b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running dd0795cc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command dd0795cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:922e35b9-25f1-4fd5-8083-0da21011ee6d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:922e35b9-25f1-4fd5-8083-0da21011ee6d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running de1f7cf4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command de1f7cf4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command de1f7cf4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4168 (Sun Sep 16 01:43:11 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4168 (Sun Sep 16 01:43:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32a005eb", "local_conf_timestamp": 4168, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Sun Sep 16 01:43:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4165 (Sun Sep 16 01:43:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "90d5648d", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:efebb507-eed1-402c-900c-d8b991d64c98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:efebb507-eed1-402c-900c-d8b991d64c98:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e54f9eb4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e54f9eb4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e54f9eb4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Sun Sep 16 01:43:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4178 (Sun Sep 16 01:43:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sun Sep 16 01:43:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4175 (Sun Sep 16 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94939b88", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3c4221fd-f368-4daa-9fd6-c7d9c62fe6e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3c4221fd-f368-4daa-9fd6-c7d9c62fe6e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running e91662f8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command e91662f8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command e91662f8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Sun Sep 16 01:43:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4178 (Sun Sep 16 01:43:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fd39986b", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Sun Sep 16 01:43:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4175 (Sun Sep 16 01:43:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94939b88", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fb8f553e-88a1-42b7-92fc-fea7ff17a32e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fb8f553e-88a1-42b7-92fc-fea7ff17a32e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ec062778 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ec062778 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ec062778 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Sun Sep 16 01:43:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Sun Sep 16 01:43:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Sep 16 01:43:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Sun Sep 16 01:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc49ab23", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:2d875d5d-1c8f-4900-a691-da5036cd0cd0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2d875d5d-1c8f-4900-a691-da5036cd0cd0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ef0b27e8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ef0b27e8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ef0b27e8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Sun Sep 16 01:43:30 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4188 (Sun Sep 16 01:43:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "708cc970", "local_conf_timestamp": 4188, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Sun Sep 16 01:43:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Sun Sep 16 01:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc49ab23", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d4dc8859-9e86-48d7-84c4-911339239213:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d4dc8859-9e86-48d7-84c4-911339239213:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f1fedfa8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f1fedfa8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f1fedfa8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Sun Sep 16 01:43:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Sun Sep 16 01:43:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Sep 16 01:43:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4195 (Sun Sep 16 01:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d80f5426", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7dee7209-2ea3-473b-b364-cb7825e74580:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7dee7209-2ea3-473b-b364-cb7825e74580:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f4e7ed72 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f4e7ed72 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f4e7ed72 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Sun Sep 16 01:43:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Sun Sep 16 01:43:41 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "debff7b2", "local_conf_timestamp": 4198, "host-ts": 4198}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4195 (Sun Sep 16 01:43:37 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4195 (Sun Sep 16 01:43:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d80f5426", "local_conf_timestamp": 4195, "host-ts": 4195}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c46089f7-dbd0-4907-b521-e7a04d8c3779:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c46089f7-dbd0-4907-b521-e7a04d8c3779:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f7d2dd4e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command f7d2dd4e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command f7d2dd4e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Sun Sep 16 01:43:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Sun Sep 16 01:43:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 16 01:43:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Sun Sep 16 01:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4b101b93-85db-4a08-b611-266c770f9b8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4b101b93-85db-4a08-b611-266c770f9b8b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running faf06460 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command faf06460 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command faf06460 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Sun Sep 16 01:43:51 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Sun Sep 16 01:43:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e808add7", "local_conf_timestamp": 4209, "host-ts": 4208}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Sun Sep 16 01:43:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4205 (Sun Sep 16 01:43:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6ecd859b", "local_conf_timestamp": 4205, "host-ts": 4205}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f9afa89c-bccb-4b59-b3ea-532ac669fdc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f9afa89c-bccb-4b59-b3ea-532ac669fdc8:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running fe38ee4e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command fe38ee4e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command fe38ee4e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Sun Sep 16 01:44:00 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Sun Sep 16 01:44:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "313ca383", "local_conf_timestamp": 4218, "host-ts": 4218}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Sun Sep 16 01:43:57 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4215 (Sun Sep 16 01:43:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d9afa210", "local_conf_timestamp": 4215, "host-ts": 4215}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:78fe242c-6aa9-4edb-9536-bb7ae9e479ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:78fe242c-6aa9-4edb-9536-bb7ae9e479ce:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 01615fb6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 01615fb6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 01615fb6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sun Sep 16 01:44:11 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sun Sep 16 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 16 01:44:07 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 16 01:44:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:762a8160-7a78-4f11-9480-671b38b5bf7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:762a8160-7a78-4f11-9480-671b38b5bf7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0503352c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0503352c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0503352c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sun Sep 16 01:44:11 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sun Sep 16 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Sun Sep 16 01:44:07 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4225 (Sun Sep 16 01:44:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d564a31f", "local_conf_timestamp": 4225, "host-ts": 4225}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ff044350-065e-485f-a443-a37aa4a70784:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ff044350-065e-485f-a443-a37aa4a70784:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 07dad32c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 07dad32c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 07dad32c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Sun Sep 16 01:44:11 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4228 (Sun Sep 16 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1818e684", "local_conf_timestamp": 4228, "host-ts": 4228}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 16 01:44:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4235 (Sun Sep 16 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "97bf29cd", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3f1e3607-915e-43f2-b21c-44cf9ff7ca22:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3f1e3607-915e-43f2-b21c-44cf9ff7ca22:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0aaf70ee on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0aaf70ee on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0aaf70ee on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Sun Sep 16 01:44:21 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4239 (Sun Sep 16 01:44:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "46234a3f", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 16 01:44:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4235 (Sun Sep 16 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "97bf29cd", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ef7bb398-b949-401a-a366-921d0ea537b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ef7bb398-b949-401a-a366-921d0ea537b1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 0bc370c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 0bc370c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 0bc370c0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Sun Sep 16 01:44:21 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4239 (Sun Sep 16 01:44:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "46234a3f", "local_conf_timestamp": 4239, "host-ts": 4238}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Sun Sep 16 01:44:17 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4235 (Sun Sep 16 01:44:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "97bf29cd", "local_conf_timestamp": 4235, "host-ts": 4235}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/317/ Build Number: 317 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.install_cockpit_ovirt Error Message: _install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host <ovirtlago.virt.HostVM object at 0x7f095774cbd0> -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:658c9895-30cd-4f66-ba11-c1e4e7b1f1a2:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: start task:ee6300c1-8ea8-476e-95ad-dfd1728a030e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:658c9895-30cd-4f66-ba11-c1e4e7b1f1a2:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:ee6300c1-8ea8-476e-95ad-dfd1728a030e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running f9c7e82e on lago-he-node-ng-suite-4-2-host-1: yum -y install cockpit-ovirt-dashboard lago.ssh: DEBUG: Running f9d09078 on lago-he-node-ng-suite-4-2-host-0: yum -y install cockpit-ovirt-dashboard lago.ssh: DEBUG: Command f9c7e82e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command f9c7e82e on lago-he-node-ng-suite-4-2-host-1 output: Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, : package_upload, product-id, search-disabled-repos, subscription- : manager, vdsmupgrade This system is not registered with an entitlement server. You can use subscription-manager to register. Determining fastest mirrors * alocalsync: 192.168.200.1:8585 * ovirt-4.2-epel: dl.fedoraproject.org Nothing to do Uploading Enabled Repositories Report Loaded plugins: fastestmirror, product-id, subscription-manager This system is not registered with an entitlement server. You can use subscription-manager to register. lago.ssh: DEBUG: Command f9c7e82e on lago-he-node-ng-suite-4-2-host-1 errors: http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. Cannot upload enabled repos report, is this client registered? lago.ssh: DEBUG: Command f9d09078 on lago-he-node-ng-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command f9d09078 on lago-he-node-ng-suite-4-2-host-0 output: Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, : package_upload, product-id, search-disabled-repos, subscription- : manager, vdsmupgrade This system is not registered with an entitlement server. You can use subscription-manager to register. Determining fastest mirrors * alocalsync: 192.168.200.1:8585 * ovirt-4.2-epel: dl.fedoraproject.org Uploading Enabled Repositories Report Loaded plugins: fastestmirror, product-id, subscription-manager This system is not registered with an entitlement server. You can use subscription-manager to register. lago.ssh: DEBUG: Command f9d09078 on lago-he-node-ng-suite-4-2-host-0 errors: BDB2053 Freeing read locks for locker 0xa6: 32348/139626396850240 BDB2053 Freeing read locks for locker 0xa8: 32348/139626396850240 BDB2053 Freeing read locks for locker 0xa9: 32348/139626396850240 BDB2053 Freeing read locks for locker 0xaa: 32348/139626396850240 http://192.168.200.1:8585/el7/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found Trying other mirror. To address this issue please refer to the below wiki article https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Trying other mirror. One of the configured repositories failed (CentOS-7 - SCLo rh), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this: 1. Contact the upstream for the repository and get them to fix the problem. 2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work). 3. Run the command with the repository temporarily disabled yum --disablerepo=centos-sclo-rh-release ... 4. Disable the repository permanently, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage: yum-config-manager --disable centos-sclo-rh-release or subscription-manager repos --disable=centos-sclo-rh-release 5. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise: yum-config-manager --save --setopt=centos-sclo-rh-release.skip_if_unavailable=true failure: repodata/repomd.xml from centos-sclo-rh-release: [Errno 256] No more mirrors to try. http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable" Cannot upload enabled repos report, is this client registered? lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/002_bootstrap.py", line 414, in _install_cockpit_ovirt_on_host nt.assert_equals(ret.code, 0, '_install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host %s' % host) File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual raise self.failureException(msg) AssertionError: _install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host <ovirtlago.virt.HostVM object at 0x7f095774cbd0> --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/002_bootstrap.py", line 421, in install_cockpit_ovirt nt.assert_true(all(vt.join_all()), 'not all threads finished: %s' % vt) File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/002_bootstrap.py", line 414, in _install_cockpit_ovirt_on_host nt.assert_equals(ret.code, 0, '_install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host %s' % host) File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual raise self.failureException(msg) '_install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host <ovirtlago.virt.HostVM object at 0x7f095774cbd0>\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:658c9895-30cd-4f66-ba11-c1e4e7b1f1a2:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: start task:ee6300c1-8ea8-476e-95ad-dfd1728a030e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:658c9895-30cd-4f66-ba11-c1e4e7b1f1a2:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:ee6300c1-8ea8-476e-95ad-dfd1728a030e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running f9c7e82e on lago-he-node-ng-suite-4-2-host-1: yum -y install cockpit-ovirt-dashboard\nlago.ssh: DEBUG: Running f9d09078 on lago-he-node-ng-suite-4-2-host-0: yum -y install cockpit-ovirt-dashboard\nlago.ssh: DEBUG: Command f9c7e82e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command f9c7e82e on lago-he-node-ng-suite-4-2-host-1 output:\n Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist,\n : package_upload, product-id, search-disabled-repos, subscription-\n : manager, vdsmupgrade\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\nDetermining fastest mirrors\n * alocalsync: 192.168.200.1:8585\n * ovirt-4.2-epel: dl.fedoraproject.org\nNothing to do\nUploading Enabled Repositories Report\nLoaded plugins: fastestmirror, product-id, subscription-manager\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\n\nlago.ssh: DEBUG: Command f9c7e82e on lago-he-node-ng-suite-4-2-host-1 errors:\n http://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nCannot upload enabled repos report, is this client registered?\n\nlago.ssh: DEBUG: Command f9d09078 on lago-he-node-ng-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command f9d09078 on lago-he-node-ng-suite-4-2-host-0 output:\n Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist,\n : package_upload, product-id, search-disabled-repos, subscription-\n : manager, vdsmupgrade\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\nDetermining fastest mirrors\n * alocalsync: 192.168.200.1:8585\n * ovirt-4.2-epel: dl.fedoraproject.org\nUploading Enabled Repositories Report\nLoaded plugins: fastestmirror, product-id, subscription-manager\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\n\nlago.ssh: DEBUG: Command f9d09078 on lago-he-node-ng-suite-4-2-host-0 errors:\n BDB2053 Freeing read locks for locker 0xa6: 32348/139626396850240\nBDB2053 Freeing read locks for locker 0xa8: 32348/139626396850240\nBDB2053 Freeing read locks for locker 0xa9: 32348/139626396850240\nBDB2053 Freeing read locks for locker 0xaa: 32348/139626396850240\nhttp://192.168.200.1:8585/el7/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found\nTrying other mirror.\nTo address this issue please refer to the below wiki article \n\nhttps://wiki.centos.org/yum-errors\n\nIf above article doesn\'t help to resolve this issue please use https://bugs.centos.org/.\n\nhttp://mirror.centos.org/centos/7/sclo/x86_6...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nTrying other mirror.\n\n\n One of the configured repositories failed (CentOS-7 - SCLo rh),\n and yum doesn\'t have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work "fix" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=centos-sclo-rh-release ...\n\n 4. Disable the repository permanently, so yum won\'t use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable centos-sclo-rh-release\n or\n subscription-manager repos --disable=centos-sclo-rh-release\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=centos-sclo-rh-release.skip_if_unavailable=true\n\nfailure: repodata/repomd.xml from centos-sclo-rh-release: [Errno 256] No more mirrors to try.\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nhttp://mirror.centos.org/centos/7/sclo/x86_64/rh/repod...: [Errno 14] curl#7 - "Failed connect to mirror.centos.org:80; Network is unreachable"\nCannot upload enabled repos report, is this client registered?\n\nlago.utils: ERROR: Error while running thread\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/002_bootstrap.py", line 414, in _install_cockpit_ovirt_on_host\n nt.assert_equals(ret.code, 0, \'_install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host %s\' % host)\n File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual\n assertion_func(first, second, msg=msg)\n File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual\n raise self.failureException(msg)\nAssertionError: _install_cockpit_ovirt_on_host(): failed to install cockpit-ovirt-dashboard on host <ovirtlago.virt.HostVM object at 0x7f095774cbd0>\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/319/ Build Number: 319 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:8efa82c2-ed02-4247-aa91-e50d6a4a0596:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:8efa82c2-ed02-4247-aa91-e50d6a4a0596:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 31d6ce44 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 31d6ce44 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 31d6ce44 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4097 (Mon Sep 17 09:43:43 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4098 (Mon Sep 17 09:43:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "30d99f17", "local_conf_timestamp": 4098, "host-ts": 4097}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4104 (Mon Sep 17 09:43:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4104 (Mon Sep 17 09:43:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149a7615", "local_conf_timestamp": 4104, "host-ts": 4104}, "global_maintenance": false} lago.ssh: DEBUG: start task:e077960f-7df4-4499-aebd-45ddba26f653:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:e077960f-7df4-4499-aebd-45ddba26f653:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 32e7b398 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 32e7b398 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:64cd8f9f-aa74-467b-a2e2-ee5d8f85cda2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:64cd8f9f-aa74-467b-a2e2-ee5d8f85cda2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3405924a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3405924a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3405924a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4108 (Mon Sep 17 09:43:53 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4108 (Mon Sep 17 09:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b50fd886", "local_conf_timestamp": 4108, "host-ts": 4108}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4104 (Mon Sep 17 09:43:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4104 (Mon Sep 17 09:43:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149a7615", "local_conf_timestamp": 4104, "host-ts": 4104}, "global_maintenance": false} lago.ssh: DEBUG: start task:12103ad7-87af-415b-ab05-3b0cfb446753:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:12103ad7-87af-415b-ab05-3b0cfb446753:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3b434214 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3b434214 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3b434214 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4118 (Mon Sep 17 09:44:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4118 (Mon Sep 17 09:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a07cf95", "local_conf_timestamp": 4118, "host-ts": 4118}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4114 (Mon Sep 17 09:43:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4114 (Mon Sep 17 09:43:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "10dc8910", "local_conf_timestamp": 4114, "host-ts": 4114}, "global_maintenance": false} lago.ssh: DEBUG: start task:d5fb38ca-5839-412a-a64c-d1a2baf5b209:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d5fb38ca-5839-412a-a64c-d1a2baf5b209:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3e4cf78e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3e4cf78e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3e4cf78e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4128 (Mon Sep 17 09:44:14 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4128 (Mon Sep 17 09:44:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "86555762", "local_conf_timestamp": 4128, "host-ts": 4128}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4124 (Mon Sep 17 09:44:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4124 (Mon Sep 17 09:44:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c17881f", "local_conf_timestamp": 4124, "host-ts": 4124}, "global_maintenance": false} lago.ssh: DEBUG: start task:1a5dc7b8-2d8f-431f-9ac5-fa5ccd6d881e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1a5dc7b8-2d8f-431f-9ac5-fa5ccd6d881e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 41b71864 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 41b71864 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 41b71864 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4138 (Mon Sep 17 09:44:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4138 (Mon Sep 17 09:44:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "286669a0", "local_conf_timestamp": 4138, "host-ts": 4138}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4134 (Mon Sep 17 09:44:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4135 (Mon Sep 17 09:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "186b7214", "local_conf_timestamp": 4135, "host-ts": 4134}, "global_maintenance": false} lago.ssh: DEBUG: start task:6a156950-4f69-4ba3-934a-144b8cdf7442:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:6a156950-4f69-4ba3-934a-144b8cdf7442:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 44b83002 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 44b83002 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 44b83002 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4138 (Mon Sep 17 09:44:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4138 (Mon Sep 17 09:44:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "286669a0", "local_conf_timestamp": 4138, "host-ts": 4138}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4134 (Mon Sep 17 09:44:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4135 (Mon Sep 17 09:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "186b7214", "local_conf_timestamp": 4135, "host-ts": 4134}, "global_maintenance": false} lago.ssh: DEBUG: start task:b7337d49-294a-460c-bebd-d3abaf7d6de1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b7337d49-294a-460c-bebd-d3abaf7d6de1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 47c69f18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 47c69f18 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 47c69f18 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4138 (Mon Sep 17 09:44:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4138 (Mon Sep 17 09:44:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "286669a0", "local_conf_timestamp": 4138, "host-ts": 4138}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4145 (Mon Sep 17 09:44:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4145 (Mon Sep 17 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false} lago.ssh: DEBUG: start task:5115a681-97a8-45dd-8f47-25dd19836b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5115a681-97a8-45dd-8f47-25dd19836b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4abc307a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4abc307a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4abc307a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4148 (Mon Sep 17 09:44:33 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4148 (Mon Sep 17 09:44:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d41ddd6c", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4145 (Mon Sep 17 09:44:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4145 (Mon Sep 17 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false} lago.ssh: DEBUG: start task:3fb7ef5d-ab18-436a-a34e-fcab47551480:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3fb7ef5d-ab18-436a-a34e-fcab47551480:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4da3ca6e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4da3ca6e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4da3ca6e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4148 (Mon Sep 17 09:44:33 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4148 (Mon Sep 17 09:44:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d41ddd6c", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4145 (Mon Sep 17 09:44:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4145 (Mon Sep 17 09:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false} lago.ssh: DEBUG: start task:f710fc96-b8d6-4724-b6d5-f1377fe125c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f710fc96-b8d6-4724-b6d5-f1377fe125c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5095ede2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5095ede2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5095ede2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4158 (Mon Sep 17 09:44:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4158 (Mon Sep 17 09:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7a2ee3ae", "local_conf_timestamp": 4158, "host-ts": 4158}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4155 (Mon Sep 17 09:44:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4155 (Mon Sep 17 09:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "58078894", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false} lago.ssh: DEBUG: start task:bed806f6-b734-4fda-89a7-212ba9af9648:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bed806f6-b734-4fda-89a7-212ba9af9648:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 53b88f34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 53b88f34 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 53b88f34 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4168 (Mon Sep 17 09:44:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4168 (Mon Sep 17 09:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "530aa6a9", "local_conf_timestamp": 4168, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Mon Sep 17 09:44:50 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4165 (Mon Sep 17 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false} lago.ssh: DEBUG: start task:915904bc-a8cd-4199-9c51-f4e8bb0b69b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:915904bc-a8cd-4199-9c51-f4e8bb0b69b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 56ed05e0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 56ed05e0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 56ed05e0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4168 (Mon Sep 17 09:44:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4168 (Mon Sep 17 09:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "530aa6a9", "local_conf_timestamp": 4168, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Mon Sep 17 09:44:50 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4165 (Mon Sep 17 09:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false} lago.ssh: DEBUG: start task:a5066fa0-cb31-4215-bb94-d6dbafef868f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a5066fa0-cb31-4215-bb94-d6dbafef868f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 59d908a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 59d908a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 59d908a8 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Mon Sep 17 09:45:03 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4178 (Mon Sep 17 09:45:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6d72dffa", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Mon Sep 17 09:45:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4175 (Mon Sep 17 09:45:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "508a769e", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:91f3ea9b-5f8b-4b1a-8640-7a9a925c6697:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:91f3ea9b-5f8b-4b1a-8640-7a9a925c6697:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5aebe49a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5aebe49a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5aebe49a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4178 (Mon Sep 17 09:45:03 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4178 (Mon Sep 17 09:45:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6d72dffa", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Mon Sep 17 09:45:00 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4175 (Mon Sep 17 09:45:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "508a769e", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:4f85f8bc-b0ce-4e38-9866-1eef5f12d6d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4f85f8bc-b0ce-4e38-9866-1eef5f12d6d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 61f0bc84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 61f0bc84 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 61f0bc84 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4188 (Mon Sep 17 09:45:14 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4189 (Mon Sep 17 09:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "fb780ac0", "local_conf_timestamp": 4189, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Mon Sep 17 09:45:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4185 (Mon Sep 17 09:45:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7d5729b6", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:8efa82c2-ed02-4247-aa91-e50d6a4a0596:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:8efa82c2-ed02-4247-aa91-e50d6a4a0596:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 31d6ce44 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 31d6ce44 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 31d6ce44 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4097 (Mon Sep 17 09:43:43 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4098 (Mon Sep 17 09:43:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "30d99f17", "local_conf_timestamp": 4098, "host-ts": 4097}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4104 (Mon Sep 17 09:43:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4104 (Mon Sep 17 09:43:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149a7615", "local_conf_timestamp": 4104, "host-ts": 4104}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:e077960f-7df4-4499-aebd-45ddba26f653:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e077960f-7df4-4499-aebd-45ddba26f653:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 32e7b398 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 32e7b398 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:64cd8f9f-aa74-467b-a2e2-ee5d8f85cda2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:64cd8f9f-aa74-467b-a2e2-ee5d8f85cda2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3405924a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3405924a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3405924a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4108 (Mon Sep 17 09:43:53 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4108 (Mon Sep 17 09:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b50fd886", "local_conf_timestamp": 4108, "host-ts": 4108}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4104 (Mon Sep 17 09:43:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4104 (Mon Sep 17 09:43:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "149a7615", "local_conf_timestamp": 4104, "host-ts": 4104}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:12103ad7-87af-415b-ab05-3b0cfb446753:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:12103ad7-87af-415b-ab05-3b0cfb446753:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3b434214 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3b434214 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3b434214 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4118 (Mon Sep 17 09:44:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4118 (Mon Sep 17 09:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4a07cf95", "local_conf_timestamp": 4118, "host-ts": 4118}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4114 (Mon Sep 17 09:43:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4114 (Mon Sep 17 09:43:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "10dc8910", "local_conf_timestamp": 4114, "host-ts": 4114}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d5fb38ca-5839-412a-a64c-d1a2baf5b209:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d5fb38ca-5839-412a-a64c-d1a2baf5b209:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3e4cf78e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3e4cf78e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3e4cf78e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4128 (Mon Sep 17 09:44:14 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4128 (Mon Sep 17 09:44:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "86555762", "local_conf_timestamp": 4128, "host-ts": 4128}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4124 (Mon Sep 17 09:44:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4124 (Mon Sep 17 09:44:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1c17881f", "local_conf_timestamp": 4124, "host-ts": 4124}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1a5dc7b8-2d8f-431f-9ac5-fa5ccd6d881e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1a5dc7b8-2d8f-431f-9ac5-fa5ccd6d881e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 41b71864 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 41b71864 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 41b71864 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4138 (Mon Sep 17 09:44:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4138 (Mon Sep 17 09:44:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "286669a0", "local_conf_timestamp": 4138, "host-ts": 4138}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4134 (Mon Sep 17 09:44:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4135 (Mon Sep 17 09:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "186b7214", "local_conf_timestamp": 4135, "host-ts": 4134}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:6a156950-4f69-4ba3-934a-144b8cdf7442:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6a156950-4f69-4ba3-934a-144b8cdf7442:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 44b83002 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 44b83002 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 44b83002 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4138 (Mon Sep 17 09:44:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4138 (Mon Sep 17 09:44:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "286669a0", "local_conf_timestamp": 4138, "host-ts": 4138}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4134 (Mon Sep 17 09:44:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4135 (Mon Sep 17 09:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "186b7214", "local_conf_timestamp": 4135, "host-ts": 4134}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b7337d49-294a-460c-bebd-d3abaf7d6de1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b7337d49-294a-460c-bebd-d3abaf7d6de1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 47c69f18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 47c69f18 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 47c69f18 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4138 (Mon Sep 17 09:44:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4138 (Mon Sep 17 09:44:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "286669a0", "local_conf_timestamp": 4138, "host-ts": 4138}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4145 (Mon Sep 17 09:44:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4145 (Mon Sep 17 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5115a681-97a8-45dd-8f47-25dd19836b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5115a681-97a8-45dd-8f47-25dd19836b45:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4abc307a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4abc307a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4abc307a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4148 (Mon Sep 17 09:44:33 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4148 (Mon Sep 17 09:44:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d41ddd6c", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4145 (Mon Sep 17 09:44:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4145 (Mon Sep 17 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3fb7ef5d-ab18-436a-a34e-fcab47551480:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3fb7ef5d-ab18-436a-a34e-fcab47551480:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4da3ca6e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4da3ca6e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4da3ca6e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4148 (Mon Sep 17 09:44:33 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4148 (Mon Sep 17 09:44:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d41ddd6c", "local_conf_timestamp": 4148, "host-ts": 4148}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4145 (Mon Sep 17 09:44:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4145 (Mon Sep 17 09:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f710fc96-b8d6-4724-b6d5-f1377fe125c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f710fc96-b8d6-4724-b6d5-f1377fe125c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5095ede2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5095ede2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5095ede2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4158 (Mon Sep 17 09:44:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4158 (Mon Sep 17 09:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7a2ee3ae", "local_conf_timestamp": 4158, "host-ts": 4158}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4155 (Mon Sep 17 09:44:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4155 (Mon Sep 17 09:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "58078894", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bed806f6-b734-4fda-89a7-212ba9af9648:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bed806f6-b734-4fda-89a7-212ba9af9648:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 53b88f34 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 53b88f34 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 53b88f34 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4168 (Mon Sep 17 09:44:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4168 (Mon Sep 17 09:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "530aa6a9", "local_conf_timestamp": 4168, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Mon Sep 17 09:44:50 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4165 (Mon Sep 17 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:915904bc-a8cd-4199-9c51-f4e8bb0b69b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:915904bc-a8cd-4199-9c51-f4e8bb0b69b7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 56ed05e0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 56ed05e0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 56ed05e0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4168 (Mon Sep 17 09:44:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4168 (Mon Sep 17 09:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "530aa6a9", "local_conf_timestamp": 4168, "host-ts": 4168}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Mon Sep 17 09:44:50 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4165 (Mon Sep 17 09:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a5066fa0-cb31-4215-bb94-d6dbafef868f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a5066fa0-cb31-4215-bb94-d6dbafef868f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 59d908a8 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 59d908a8 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 59d908a8 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Mon Sep 17 09:45:03 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4178 (Mon Sep 17 09:45:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6d72dffa", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Mon Sep 17 09:45:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4175 (Mon Sep 17 09:45:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "508a769e", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:91f3ea9b-5f8b-4b1a-8640-7a9a925c6697:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:91f3ea9b-5f8b-4b1a-8640-7a9a925c6697:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5aebe49a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5aebe49a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5aebe49a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4178 (Mon Sep 17 09:45:03 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4178 (Mon Sep 17 09:45:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6d72dffa", "local_conf_timestamp": 4178, "host-ts": 4178}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Mon Sep 17 09:45:00 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4175 (Mon Sep 17 09:45:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "508a769e", "local_conf_timestamp": 4175, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4f85f8bc-b0ce-4e38-9866-1eef5f12d6d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4f85f8bc-b0ce-4e38-9866-1eef5f12d6d7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 61f0bc84 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 61f0bc84 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 61f0bc84 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4188 (Mon Sep 17 09:45:14 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4189 (Mon Sep 17 09:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "fb780ac0", "local_conf_timestamp": 4189, "host-ts": 4188}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Mon Sep 17 09:45:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4185 (Mon Sep 17 09:45:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7d5729b6", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/321/ Build Number: 321 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 5fd253d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false} lago.ssh: DEBUG: start task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 60dd7c52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 60dd7c52 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 61fe1d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false} lago.ssh: DEBUG: start task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6a9e0004 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6d936038 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 708dde08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 73876c82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 769e011a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 799ec070 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7ca252a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false} lago.ssh: DEBUG: start task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7fe373fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false} lago.ssh: DEBUG: start task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 834fc6fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false} lago.ssh: DEBUG: start task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 86917a20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false} lago.ssh: DEBUG: start task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 89e318d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4288 (Tue Sep 18 01:45:33 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4288 (Tue Sep 18 01:45:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f547c6f", "local_conf_timestamp": 4288, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4280 (Tue Sep 18 01:45:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4280 (Tue Sep 18 01:45:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78f0fd60", "local_conf_timestamp": 4280, "host-ts": 4280}, "global_maintenance": false} lago.ssh: DEBUG: start task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8af57904 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 40, in _wait_for_engine_migration lambda: _check_migration_state(host, state) is False File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 40, in <lambda> lambda: _check_migration_state(host, state) is False File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 103, in _check_migration_state status = _get_he_status(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 92, in _get_he_status ret = host.ssh(["hosted-engine", "--vm-status", "--json"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 69, in ssh 'stderr': None File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 247, in drain_ssh_channel 0.1, File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 5fd253d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 60dd7c52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 60dd7c52 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 61fe1d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6a9e0004 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6d936038 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 708dde08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 73876c82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 769e011a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 799ec070 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7ca252a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7fe373fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 834fc6fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 86917a20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 89e318d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4288 (Tue Sep 18 01:45:33 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4288 (Tue Sep 18 01:45:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f547c6f", "local_conf_timestamp": 4288, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4280 (Tue Sep 18 01:45:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4280 (Tue Sep 18 01:45:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78f0fd60", "local_conf_timestamp": 4280, "host-ts": 4280}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8af57904 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/323/ Build Number: 323 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/325/ Build Number: 325 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.clear_global_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 76, in clear_global_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/327/ Build Number: 327 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 8909c558 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "35612a16", "local_conf_timestamp": 4156, "host-ts": 4156}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false} lago.ssh: DEBUG: start task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a198168 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 8a198168 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4167 (Thu Sep 20 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4167 (Thu Sep 20 01:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "189b4b14", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4166 (Thu Sep 20 01:43:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4166 (Thu Sep 20 01:43:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "edcf5346", "local_conf_timestamp": 4166, "host-ts": 4166}, "global_maintenance": false} lago.ssh: DEBUG: start task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9302328e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false} lago.ssh: DEBUG: start task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 970c7d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false} lago.ssh: DEBUG: start task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9a10bc30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4187 (Thu Sep 20 01:43:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4187 (Thu Sep 20 01:43:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cdd0276f", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4186 (Thu Sep 20 01:43:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4186 (Thu Sep 20 01:43:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false} lago.ssh: DEBUG: start task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9d8302f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} lago.ssh: DEBUG: start task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a09033b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} lago.ssh: DEBUG: start task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a389cf0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a732a40a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aa7d047a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ad708288 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b074c5c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b189aa8e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b89697e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Thu Sep 20 01:44:27 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4237 (Thu Sep 20 01:44:27 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "055173ed", "local_conf_timestamp": 4237, "host-ts": 4237}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Thu Sep 20 01:44:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4236 (Thu Sep 20 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f183ffe", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 8909c558 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "35612a16", "local_conf_timestamp": 4156, "host-ts": 4156}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a198168 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 8a198168 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4167 (Thu Sep 20 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4167 (Thu Sep 20 01:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "189b4b14", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4166 (Thu Sep 20 01:43:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4166 (Thu Sep 20 01:43:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "edcf5346", "local_conf_timestamp": 4166, "host-ts": 4166}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9302328e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 970c7d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9a10bc30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4187 (Thu Sep 20 01:43:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4187 (Thu Sep 20 01:43:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cdd0276f", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4186 (Thu Sep 20 01:43:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4186 (Thu Sep 20 01:43:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9d8302f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a09033b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a389cf0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a732a40a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aa7d047a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ad708288 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b074c5c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b189aa8e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b89697e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Thu Sep 20 01:44:27 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4237 (Thu Sep 20 01:44:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "055173ed", "local_conf_timestamp": 4237, "host-ts": 4237}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Thu Sep 20 01:44:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4236 (Thu Sep 20 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f183ffe", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/329/ Build Number: 329 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.restart_he_vm Error Message: 143 -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:a2a0dbbb-e23d-4d73-8cb5-c502cef73da0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:a2a0dbbb-e23d-4d73-8cb5-c502cef73da0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 2c2a8138 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2c2a8138 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 2c2a8138 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3286 (Thu Sep 20 19:34:14 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3286 (Thu Sep 20 19:34:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b144a75d", "local_conf_timestamp": 3286, "host-ts": 3286}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3287 (Thu Sep 20 19:34:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3287 (Thu Sep 20 19:34:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b13e378d", "local_conf_timestamp": 3287, "host-ts": 3287}, "global_maintenance": true} root: INFO: * Shutting down HE VM on host: lago-he-node-ng-suite-4-2-host-0 lago.ssh: DEBUG: start task:46dee9c1-b9ce-4374-ae51-a4c6825052ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:46dee9c1-b9ce-4374-ae51-a4c6825052ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2d3b54da on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-shutdown lago.ssh: DEBUG: Command 2d3b54da on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Command succeeded root: INFO: * Waiting for VM to be down... lago.ssh: DEBUG: start task:fe3f2b0f-f956-4f8d-9c5a-f05fbfe9f337:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fe3f2b0f-f956-4f8d-9c5a-f05fbfe9f337:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2e941498 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2e941498 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2e941498 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3296 (Thu Sep 20 19:34:23 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3296 (Thu Sep 20 19:34:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6d23e762", "local_conf_timestamp": 3296, "host-ts": 3296}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3287 (Thu Sep 20 19:34:14 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3287 (Thu Sep 20 19:34:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b13e378d", "local_conf_timestamp": 3287, "host-ts": 3287}, "global_maintenance": true} lago.ssh: DEBUG: start task:dc9b8af4-b79a-40de-ad94-1de381e3c2b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dc9b8af4-b79a-40de-ad94-1de381e3c2b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 36a6de9a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 36a6de9a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 36a6de9a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3306 (Thu Sep 20 19:34:33 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3306 (Thu Sep 20 19:34:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "711ea4b1", "local_conf_timestamp": 3306, "host-ts": 3306}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3297 (Thu Sep 20 19:34:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3297 (Thu Sep 20 19:34:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b578c888", "local_conf_timestamp": 3297, "host-ts": 3297}, "global_maintenance": true} lago.ssh: DEBUG: start task:ada919d2-9ff3-4827-9686-67b7b49e9e14:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ada919d2-9ff3-4827-9686-67b7b49e9e14:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3996199a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3996199a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3996199a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3316 (Thu Sep 20 19:34:44 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3316 (Thu Sep 20 19:34:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4ee0b2cb", "local_conf_timestamp": 3316, "host-ts": 3316}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3307 (Thu Sep 20 19:34:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3307 (Thu Sep 20 19:34:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e3677dd5", "local_conf_timestamp": 3307, "host-ts": 3307}, "global_maintenance": true} root: INFO: * VM is down. root: INFO: * Stopping services... lago.ssh: DEBUG: start task:9c091d09-a882-4ffb-a20e-ba1aa4d1e067:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:9c091d09-a882-4ffb-a20e-ba1aa4d1e067:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3ab481cc on lago-he-node-ng-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command 3ab481cc on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Starting services... lago.ssh: DEBUG: start task:8a28d873-3bb7-478e-ad38-3c1aa61231bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8a28d873-3bb7-478e-ad38-3c1aa61231bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 451b6eb4 on lago-he-node-ng-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 51, in restart_he_vm _restart_services(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/008_restart_he_vm.py", line 154, in _restart_services ret = host.ssh(["systemctl", "start", "vdsmd", "ovirt-ha-broker", "ovirt-ha-agent"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 69, in ssh 'stderr': None File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 247, in drain_ssh_channel 0.1, File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:a2a0dbbb-e23d-4d73-8cb5-c502cef73da0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:a2a0dbbb-e23d-4d73-8cb5-c502cef73da0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 2c2a8138 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2c2a8138 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 2c2a8138 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3286 (Thu Sep 20 19:34:14 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3286 (Thu Sep 20 19:34:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b144a75d", "local_conf_timestamp": 3286, "host-ts": 3286}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3287 (Thu Sep 20 19:34:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3287 (Thu Sep 20 19:34:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b13e378d", "local_conf_timestamp": 3287, "host-ts": 3287}, "global_maintenance": true}\n\nroot: INFO: * Shutting down HE VM on host: lago-he-node-ng-suite-4-2-host-0\nlago.ssh: DEBUG: start task:46dee9c1-b9ce-4374-ae51-a4c6825052ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:46dee9c1-b9ce-4374-ae51-a4c6825052ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2d3b54da on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-shutdown\nlago.ssh: DEBUG: Command 2d3b54da on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be down...\nlago.ssh: DEBUG: start task:fe3f2b0f-f956-4f8d-9c5a-f05fbfe9f337:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fe3f2b0f-f956-4f8d-9c5a-f05fbfe9f337:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2e941498 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2e941498 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2e941498 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3296 (Thu Sep 20 19:34:23 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3296 (Thu Sep 20 19:34:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "6d23e762", "local_conf_timestamp": 3296, "host-ts": 3296}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3287 (Thu Sep 20 19:34:14 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3287 (Thu Sep 20 19:34:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b13e378d", "local_conf_timestamp": 3287, "host-ts": 3287}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:dc9b8af4-b79a-40de-ad94-1de381e3c2b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dc9b8af4-b79a-40de-ad94-1de381e3c2b3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 36a6de9a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 36a6de9a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 36a6de9a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3306 (Thu Sep 20 19:34:33 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3306 (Thu Sep 20 19:34:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "711ea4b1", "local_conf_timestamp": 3306, "host-ts": 3306}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3297 (Thu Sep 20 19:34:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3297 (Thu Sep 20 19:34:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b578c888", "local_conf_timestamp": 3297, "host-ts": 3297}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:ada919d2-9ff3-4827-9686-67b7b49e9e14:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ada919d2-9ff3-4827-9686-67b7b49e9e14:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3996199a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3996199a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3996199a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3316 (Thu Sep 20 19:34:44 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3316 (Thu Sep 20 19:34:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "4ee0b2cb", "local_conf_timestamp": 3316, "host-ts": 3316}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3307 (Thu Sep 20 19:34:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3307 (Thu Sep 20 19:34:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e3677dd5", "local_conf_timestamp": 3307, "host-ts": 3307}, "global_maintenance": true}\n\nroot: INFO: * VM is down.\nroot: INFO: * Stopping services...\nlago.ssh: DEBUG: start task:9c091d09-a882-4ffb-a20e-ba1aa4d1e067:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9c091d09-a882-4ffb-a20e-ba1aa4d1e067:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3ab481cc on lago-he-node-ng-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command 3ab481cc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Starting services...\nlago.ssh: DEBUG: start task:8a28d873-3bb7-478e-ad38-3c1aa61231bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8a28d873-3bb7-478e-ad38-3c1aa61231bc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 451b6eb4 on lago-he-node-ng-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/331/ Build Number: 331 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 116, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/333/ Build Number: 333 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- No tests ran.

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/335/ Build Number: 335 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #334 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #335 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/337/ Build Number: 337 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #334 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #335 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #336 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #337 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/339/ Build Number: 339 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #334 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #335 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #336 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #337 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #338 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #339 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/341/ Build Number: 341 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #334 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #335 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #336 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #337 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #338 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #339 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #340 [Sahina Bose] hc: Fixing ansible version for 4.2 as well [Martin Perina] Add ovirt-engine-wildfly to STDCIv2 [Martin Perina] Remove STDCIv1 jobs for ovirt-engine-wildfly [Martin Perina] Add ovirt-engine-wildfly-overlay to STDCIv2 [Martin Perina] Remove STDCIv1 jobs for ovirt-engine-wildfly-overlay Changes for Build #341 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:f977a8ee-1df7-4deb-812d-f22325126ff8:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:f977a8ee-1df7-4deb-812d-f22325126ff8:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 1456abaa on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1456abaa on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 1456abaa on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4169 (Mon Sep 24 17:41:20 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4170 (Mon Sep 24 17:41:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c0452b0c", "local_conf_timestamp": 4170, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Mon Sep 24 17:41:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4177 (Mon Sep 24 17:41:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7450bcc5", "local_conf_timestamp": 4177, "host-ts": 4177}, "global_maintenance": false} lago.ssh: DEBUG: start task:db13d413-3d5b-4c07-bcc6-da7dcdac543a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:db13d413-3d5b-4c07-bcc6-da7dcdac543a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1567a490 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 1567a490 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:2b62d332-6411-411a-94ef-d56779a837b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:2b62d332-6411-411a-94ef-d56779a837b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 167bf3c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 167bf3c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 167bf3c2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4179 (Mon Sep 24 17:41:29 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4179 (Mon Sep 24 17:41:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7ce5e2d6", "local_conf_timestamp": 4179, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Mon Sep 24 17:41:27 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4177 (Mon Sep 24 17:41:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7450bcc5", "local_conf_timestamp": 4177, "host-ts": 4177}, "global_maintenance": false} lago.ssh: DEBUG: start task:623c6067-ecde-4219-9727-70c5250dabcf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:623c6067-ecde-4219-9727-70c5250dabcf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 1d94dd68 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 1d94dd68 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 1d94dd68 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Mon Sep 24 17:41:50 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4200 (Mon Sep 24 17:41:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "96b3c2ee", "local_conf_timestamp": 4200, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Mon Sep 24 17:41:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4198 (Mon Sep 24 17:41:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1df2a855", "local_conf_timestamp": 4198, "host-ts": 4198}, "global_maintenance": false} lago.ssh: DEBUG: start task:40e78aba-e44e-4371-9919-c4a2930355dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:40e78aba-e44e-4371-9919-c4a2930355dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 20c5beb2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 20c5beb2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 20c5beb2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4199 (Mon Sep 24 17:41:50 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4200 (Mon Sep 24 17:41:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "96b3c2ee", "local_conf_timestamp": 4200, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4198 (Mon Sep 24 17:41:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4198 (Mon Sep 24 17:41:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1df2a855", "local_conf_timestamp": 4198, "host-ts": 4198}, "global_maintenance": false} lago.ssh: DEBUG: start task:85ca5145-e1ad-43bf-a2f4-a77c73be9721:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:85ca5145-e1ad-43bf-a2f4-a77c73be9721:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 23c8e99a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 23c8e99a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 23c8e99a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Mon Sep 24 17:41:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Mon Sep 24 17:41:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Mon Sep 24 17:41:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4208 (Mon Sep 24 17:41:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc0d4c70", "local_conf_timestamp": 4208, "host-ts": 4208}, "global_maintenance": false} lago.ssh: DEBUG: start task:bfdba066-1007-4e2b-8b59-1be528da73af:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bfdba066-1007-4e2b-8b59-1be528da73af:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 26f9feba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 26f9feba on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 26f9feba on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4209 (Mon Sep 24 17:41:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4209 (Mon Sep 24 17:41:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4208 (Mon Sep 24 17:41:58 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4208 (Mon Sep 24 17:41:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc0d4c70", "local_conf_timestamp": 4208, "host-ts": 4208}, "global_maintenance": false} lago.ssh: DEBUG: start task:4989f639-c1be-4e7d-980c-50b93d36789d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4989f639-c1be-4e7d-980c-50b93d36789d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 29eedaa0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 29eedaa0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 29eedaa0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Mon Sep 24 17:42:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4219 (Mon Sep 24 17:42:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3682b6b", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Mon Sep 24 17:42:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4218 (Mon Sep 24 17:42:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d84bb375", "local_conf_timestamp": 4218, "host-ts": 4218}, "global_maintenance": false} lago.ssh: DEBUG: start task:789ed7f6-c2e2-42b8-8252-536e7a67844a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:789ed7f6-c2e2-42b8-8252-536e7a67844a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2ced4e62 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2ced4e62 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2ced4e62 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4219 (Mon Sep 24 17:42:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4219 (Mon Sep 24 17:42:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3682b6b", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4218 (Mon Sep 24 17:42:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4218 (Mon Sep 24 17:42:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d84bb375", "local_conf_timestamp": 4218, "host-ts": 4218}, "global_maintenance": false} lago.ssh: DEBUG: start task:c54ac6a1-c5bf-4500-93ab-e11aa9b6d073:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c54ac6a1-c5bf-4500-93ab-e11aa9b6d073:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 2fe2307e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 2fe2307e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 2fe2307e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Mon Sep 24 17:42:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4229 (Mon Sep 24 17:42:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fa4c6e6c", "local_conf_timestamp": 4229, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Mon Sep 24 17:42:18 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4228 (Mon Sep 24 17:42:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67a46af4", "local_conf_timestamp": 4228, "host-ts": 4228}, "global_maintenance": false} lago.ssh: DEBUG: start task:a39845f8-451a-4205-b781-225a75944f38:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a39845f8-451a-4205-b781-225a75944f38:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 32d4a474 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 32d4a474 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 32d4a474 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4229 (Mon Sep 24 17:42:19 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4229 (Mon Sep 24 17:42:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fa4c6e6c", "local_conf_timestamp": 4229, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4228 (Mon Sep 24 17:42:18 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4228 (Mon Sep 24 17:42:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67a46af4", "local_conf_timestamp": 4228, "host-ts": 4228}, "global_maintenance": false} lago.ssh: DEBUG: start task:5df78fed-ad1e-410f-a165-24e35596001b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:5df78fed-ad1e-410f-a165-24e35596001b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3623fa3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3623fa3a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3623fa3a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4239 (Mon Sep 24 17:42:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4239 (Mon Sep 24 17:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "547f50ae", "local_conf_timestamp": 4239, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Mon Sep 24 17:42:28 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4238 (Mon Sep 24 17:42:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63e295f1", "local_conf_timestamp": 4238, "host-ts": 4238}, "global_maintenance": false} lago.ssh: DEBUG: start task:c4e84133-217b-4ad4-b772-c7765d920388:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c4e84133-217b-4ad4-b772-c7765d920388:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 393290ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 393290ec on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 393290ec on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4239 (Mon Sep 24 17:42:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4239 (Mon Sep 24 17:42:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "547f50ae", "local_conf_timestamp": 4239, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4238 (Mon Sep 24 17:42:28 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4238 (Mon Sep 24 17:42:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63e295f1", "local_conf_timestamp": 4238, "host-ts": 4238}, "global_maintenance": false} lago.ssh: DEBUG: start task:c56609b9-d09d-4231-b562-9cf188f9a3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c56609b9-d09d-4231-b562-9cf188f9a3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3c5cd994 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3c5cd994 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3c5cd994 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4249 (Mon Sep 24 17:42:39 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fb8c1fcf", "local_conf_timestamp": 4249, "host-ts": 4249}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Mon Sep 24 17:42:38 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0935587c", "local_conf_timestamp": 4249, "host-ts": 4248}, "global_maintenance": false} lago.ssh: DEBUG: start task:f874f9d9-300e-4c60-aefb-62b0c3ba6a9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f874f9d9-300e-4c60-aefb-62b0c3ba6a9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 3d7b577e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 3d7b577e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 3d7b577e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4249 (Mon Sep 24 17:42:39 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fb8c1fcf", "local_conf_timestamp": 4249, "host-ts": 4249}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Mon Sep 24 17:42:38 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0935587c", "local_conf_timestamp": 4249, "host-ts": 4248}, "global_maintenance": false} lago.ssh: DEBUG: start task:759a2d8d-ac2f-4263-9873-b8992017c0ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:759a2d8d-ac2f-4263-9873-b8992017c0ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 448dc38a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 448dc38a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 448dc38a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4259 (Mon Sep 24 17:42:49 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4259 (Mon Sep 24 17:42:50 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "cacfc882", "local_conf_timestamp": 4259, "host-ts": 4259}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Mon Sep 24 17:42:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4258 (Mon Sep 24 17:42:48 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a0ae0b6e", "local_conf_timestamp": 4258, "host-ts": 4258}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:f977a8ee-1df7-4deb-812d-f22325126ff8:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:f977a8ee-1df7-4deb-812d-f22325126ff8:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 1456abaa on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1456abaa on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 1456abaa on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4169 (Mon Sep 24 17:41:20 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4170 (Mon Sep 24 17:41:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c0452b0c", "local_conf_timestamp": 4170, "host-ts": 4169}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Mon Sep 24 17:41:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4177 (Mon Sep 24 17:41:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7450bcc5", "local_conf_timestamp": 4177, "host-ts": 4177}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:db13d413-3d5b-4c07-bcc6-da7dcdac543a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:db13d413-3d5b-4c07-bcc6-da7dcdac543a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1567a490 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 1567a490 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:2b62d332-6411-411a-94ef-d56779a837b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:2b62d332-6411-411a-94ef-d56779a837b9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 167bf3c2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 167bf3c2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 167bf3c2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4179 (Mon Sep 24 17:41:29 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4179 (Mon Sep 24 17:41:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7ce5e2d6", "local_conf_timestamp": 4179, "host-ts": 4179}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Mon Sep 24 17:41:27 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4177 (Mon Sep 24 17:41:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "7450bcc5", "local_conf_timestamp": 4177, "host-ts": 4177}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:623c6067-ecde-4219-9727-70c5250dabcf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:623c6067-ecde-4219-9727-70c5250dabcf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 1d94dd68 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 1d94dd68 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 1d94dd68 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Mon Sep 24 17:41:50 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4200 (Mon Sep 24 17:41:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "96b3c2ee", "local_conf_timestamp": 4200, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Mon Sep 24 17:41:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4198 (Mon Sep 24 17:41:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1df2a855", "local_conf_timestamp": 4198, "host-ts": 4198}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:40e78aba-e44e-4371-9919-c4a2930355dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:40e78aba-e44e-4371-9919-c4a2930355dc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 20c5beb2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 20c5beb2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 20c5beb2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4199 (Mon Sep 24 17:41:50 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4200 (Mon Sep 24 17:41:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "96b3c2ee", "local_conf_timestamp": 4200, "host-ts": 4199}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4198 (Mon Sep 24 17:41:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4198 (Mon Sep 24 17:41:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "1df2a855", "local_conf_timestamp": 4198, "host-ts": 4198}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:85ca5145-e1ad-43bf-a2f4-a77c73be9721:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:85ca5145-e1ad-43bf-a2f4-a77c73be9721:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 23c8e99a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 23c8e99a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 23c8e99a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Mon Sep 24 17:41:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Mon Sep 24 17:41:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Mon Sep 24 17:41:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4208 (Mon Sep 24 17:41:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc0d4c70", "local_conf_timestamp": 4208, "host-ts": 4208}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bfdba066-1007-4e2b-8b59-1be528da73af:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bfdba066-1007-4e2b-8b59-1be528da73af:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 26f9feba on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 26f9feba on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 26f9feba on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4209 (Mon Sep 24 17:41:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4209 (Mon Sep 24 17:41:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7d5b15a9", "local_conf_timestamp": 4209, "host-ts": 4209}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4208 (Mon Sep 24 17:41:58 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4208 (Mon Sep 24 17:41:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "dc0d4c70", "local_conf_timestamp": 4208, "host-ts": 4208}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4989f639-c1be-4e7d-980c-50b93d36789d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4989f639-c1be-4e7d-980c-50b93d36789d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 29eedaa0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 29eedaa0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 29eedaa0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Mon Sep 24 17:42:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4219 (Mon Sep 24 17:42:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3682b6b", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Mon Sep 24 17:42:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4218 (Mon Sep 24 17:42:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d84bb375", "local_conf_timestamp": 4218, "host-ts": 4218}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:789ed7f6-c2e2-42b8-8252-536e7a67844a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:789ed7f6-c2e2-42b8-8252-536e7a67844a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2ced4e62 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2ced4e62 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2ced4e62 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4219 (Mon Sep 24 17:42:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4219 (Mon Sep 24 17:42:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "d3682b6b", "local_conf_timestamp": 4219, "host-ts": 4219}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4218 (Mon Sep 24 17:42:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4218 (Mon Sep 24 17:42:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d84bb375", "local_conf_timestamp": 4218, "host-ts": 4218}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c54ac6a1-c5bf-4500-93ab-e11aa9b6d073:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c54ac6a1-c5bf-4500-93ab-e11aa9b6d073:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 2fe2307e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 2fe2307e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 2fe2307e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Mon Sep 24 17:42:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4229 (Mon Sep 24 17:42:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fa4c6e6c", "local_conf_timestamp": 4229, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Mon Sep 24 17:42:18 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4228 (Mon Sep 24 17:42:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67a46af4", "local_conf_timestamp": 4228, "host-ts": 4228}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a39845f8-451a-4205-b781-225a75944f38:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a39845f8-451a-4205-b781-225a75944f38:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 32d4a474 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 32d4a474 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 32d4a474 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4229 (Mon Sep 24 17:42:19 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4229 (Mon Sep 24 17:42:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fa4c6e6c", "local_conf_timestamp": 4229, "host-ts": 4229}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4228 (Mon Sep 24 17:42:18 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4228 (Mon Sep 24 17:42:18 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "67a46af4", "local_conf_timestamp": 4228, "host-ts": 4228}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:5df78fed-ad1e-410f-a165-24e35596001b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5df78fed-ad1e-410f-a165-24e35596001b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3623fa3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3623fa3a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3623fa3a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4239 (Mon Sep 24 17:42:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4239 (Mon Sep 24 17:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "547f50ae", "local_conf_timestamp": 4239, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Mon Sep 24 17:42:28 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4238 (Mon Sep 24 17:42:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63e295f1", "local_conf_timestamp": 4238, "host-ts": 4238}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c4e84133-217b-4ad4-b772-c7765d920388:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c4e84133-217b-4ad4-b772-c7765d920388:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 393290ec on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 393290ec on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 393290ec on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4239 (Mon Sep 24 17:42:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4239 (Mon Sep 24 17:42:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "547f50ae", "local_conf_timestamp": 4239, "host-ts": 4239}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4238 (Mon Sep 24 17:42:28 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4238 (Mon Sep 24 17:42:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "63e295f1", "local_conf_timestamp": 4238, "host-ts": 4238}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c56609b9-d09d-4231-b562-9cf188f9a3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c56609b9-d09d-4231-b562-9cf188f9a3be:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3c5cd994 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3c5cd994 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3c5cd994 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4249 (Mon Sep 24 17:42:39 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fb8c1fcf", "local_conf_timestamp": 4249, "host-ts": 4249}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Mon Sep 24 17:42:38 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0935587c", "local_conf_timestamp": 4249, "host-ts": 4248}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f874f9d9-300e-4c60-aefb-62b0c3ba6a9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f874f9d9-300e-4c60-aefb-62b0c3ba6a9f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 3d7b577e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 3d7b577e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 3d7b577e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4249 (Mon Sep 24 17:42:39 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fb8c1fcf", "local_conf_timestamp": 4249, "host-ts": 4249}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Mon Sep 24 17:42:38 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4249 (Mon Sep 24 17:42:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0935587c", "local_conf_timestamp": 4249, "host-ts": 4248}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:759a2d8d-ac2f-4263-9873-b8992017c0ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:759a2d8d-ac2f-4263-9873-b8992017c0ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 448dc38a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 448dc38a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 448dc38a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4259 (Mon Sep 24 17:42:49 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4259 (Mon Sep 24 17:42:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "cacfc882", "local_conf_timestamp": 4259, "host-ts": 4259}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Mon Sep 24 17:42:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4258 (Mon Sep 24 17:42:48 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a0ae0b6e", "local_conf_timestamp": 4258, "host-ts": 4258}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/343/ Build Number: 343 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #334 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #335 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #336 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #337 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #338 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #339 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #340 [Sahina Bose] hc: Fixing ansible version for 4.2 as well [Martin Perina] Add ovirt-engine-wildfly to STDCIv2 [Martin Perina] Remove STDCIv1 jobs for ovirt-engine-wildfly [Martin Perina] Add ovirt-engine-wildfly-overlay to STDCIv2 [Martin Perina] Remove STDCIv1 jobs for ovirt-engine-wildfly-overlay Changes for Build #341 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #342 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #343 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 012_local_maintenance_sdk.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 42, in local_maintenance time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/345/ Build Number: 345 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #328 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #329 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails Changes for Build #330 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #331 [Dominik Holler] network: Enforce specific version of ansible Changes for Build #332 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #333 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #334 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #335 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #336 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #337 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #338 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #339 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #340 [Sahina Bose] hc: Fixing ansible version for 4.2 as well [Martin Perina] Add ovirt-engine-wildfly to STDCIv2 [Martin Perina] Remove STDCIv1 jobs for ovirt-engine-wildfly [Martin Perina] Add ovirt-engine-wildfly-overlay to STDCIv2 [Martin Perina] Remove STDCIv1 jobs for ovirt-engine-wildfly-overlay Changes for Build #341 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #342 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #343 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #344 [Sahina Bose] hc: Fixing ansible version for 4.2 as well Changes for Build #345 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:941c8579-9b8f-4691-a6d8-e6fa23801812:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:941c8579-9b8f-4691-a6d8-e6fa23801812:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 966dd6ec on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 966dd6ec on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 966dd6ec on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4174 (Wed Sep 26 01:43:22 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4174 (Wed Sep 26 01:43:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Wed Sep 26 01:43:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Wed Sep 26 01:43:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cc99e1ab", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false} lago.ssh: DEBUG: start task:d7a66b41-a522-48f5-8a1f-8e8fb9f32468:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d7a66b41-a522-48f5-8a1f-8e8fb9f32468:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 977b6b26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 977b6b26 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:d6538d87-fdca-41a0-88fd-4f09820ccbfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d6538d87-fdca-41a0-88fd-4f09820ccbfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 98e96774 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 98e96774 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 98e96774 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4184 (Wed Sep 26 01:43:32 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4184 (Wed Sep 26 01:43:32 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c21c9ef7", "local_conf_timestamp": 4184, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4182 (Wed Sep 26 01:43:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4182 (Wed Sep 26 01:43:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cc99e1ab", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false} lago.ssh: DEBUG: start task:08a16e5f-0a15-455e-857b-a7420babf805:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:08a16e5f-0a15-455e-857b-a7420babf805:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a022baa4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a022baa4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a022baa4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Wed Sep 26 01:43:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Wed Sep 26 01:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Wed Sep 26 01:43:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4203 (Wed Sep 26 01:43:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:1a8bffad-bd7d-41cc-8bc8-bd7b3d551b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:1a8bffad-bd7d-41cc-8bc8-bd7b3d551b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a3170d28 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a3170d28 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a3170d28 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4205 (Wed Sep 26 01:43:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4205 (Wed Sep 26 01:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4203 (Wed Sep 26 01:43:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4203 (Wed Sep 26 01:43:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false} lago.ssh: DEBUG: start task:392c5ef3-b03b-49c3-91a9-195963d2b626:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:392c5ef3-b03b-49c3-91a9-195963d2b626:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a67cd42a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a67cd42a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a67cd42a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Wed Sep 26 01:44:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Wed Sep 26 01:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Wed Sep 26 01:44:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4213 (Wed Sep 26 01:44:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "90bf1508", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:95603aa2-ecc9-48ed-ae43-8061fc3135c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:95603aa2-ecc9-48ed-ae43-8061fc3135c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a98e6ec6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a98e6ec6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a98e6ec6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Wed Sep 26 01:44:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4216 (Wed Sep 26 01:44:04 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4213 (Wed Sep 26 01:44:00 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4213 (Wed Sep 26 01:44:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "90bf1508", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false} lago.ssh: DEBUG: start task:66b9b6bb-1c79-4dda-bc85-906e55c3c37a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:66b9b6bb-1c79-4dda-bc85-906e55c3c37a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ac879efe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ac879efe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ac879efe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Wed Sep 26 01:44:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Wed Sep 26 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Wed Sep 26 01:44:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4223 (Wed Sep 26 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:22298ee9-d865-4828-9cf7-ff814c857a68:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:22298ee9-d865-4828-9cf7-ff814c857a68:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running afcd47d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command afcd47d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command afcd47d0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4225 (Wed Sep 26 01:44:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4225 (Wed Sep 26 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4223 (Wed Sep 26 01:44:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4223 (Wed Sep 26 01:44:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false} lago.ssh: DEBUG: start task:795c2269-a0b6-4929-a1a3-5d7b5d594a0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:795c2269-a0b6-4929-a1a3-5d7b5d594a0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b2e4baf2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b2e4baf2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b2e4baf2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Wed Sep 26 01:44:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Wed Sep 26 01:44:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Wed Sep 26 01:44:21 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4233 (Wed Sep 26 01:44:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:7d3fba27-a87e-4f7e-8387-0abb7f959531:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7d3fba27-a87e-4f7e-8387-0abb7f959531:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b5fcdfbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b5fcdfbc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b5fcdfbc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4235 (Wed Sep 26 01:44:24 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4236 (Wed Sep 26 01:44:24 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4233 (Wed Sep 26 01:44:21 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4233 (Wed Sep 26 01:44:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false} lago.ssh: DEBUG: start task:d2a7fced-d909-40a6-9f7f-0d04cd7dc85d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:d2a7fced-d909-40a6-9f7f-0d04cd7dc85d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b972878c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b972878c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b972878c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Wed Sep 26 01:44:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4246 (Wed Sep 26 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "15580a7d", "local_conf_timestamp": 4246, "host-ts": 4246}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Wed Sep 26 01:44:31 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4244 (Wed Sep 26 01:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a8a25b34", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false} lago.ssh: DEBUG: start task:04fcaf5e-6061-47d1-8e5f-f189a1a2ba80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:04fcaf5e-6061-47d1-8e5f-f189a1a2ba80:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running bce6191a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command bce6191a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command bce6191a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4246 (Wed Sep 26 01:44:34 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4246 (Wed Sep 26 01:44:34 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "15580a7d", "local_conf_timestamp": 4246, "host-ts": 4246}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4243 (Wed Sep 26 01:44:31 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4244 (Wed Sep 26 01:44:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a8a25b34", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false} lago.ssh: DEBUG: start task:f780d778-0bd0-4148-a9fb-484d8aa7e20b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f780d778-0bd0-4148-a9fb-484d8aa7e20b:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c032e47c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c032e47c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c032e47c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Wed Sep 26 01:44:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4256 (Wed Sep 26 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b158234b", "local_conf_timestamp": 4256, "host-ts": 4256}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Wed Sep 26 01:44:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4253 (Wed Sep 26 01:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32803192", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false} lago.ssh: DEBUG: start task:fba632ca-9225-448b-a3d4-97e1026e0168:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:fba632ca-9225-448b-a3d4-97e1026e0168:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c14b6d8e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c14b6d8e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c14b6d8e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4256 (Wed Sep 26 01:44:44 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4256 (Wed Sep 26 01:44:44 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b158234b", "local_conf_timestamp": 4256, "host-ts": 4256}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4253 (Wed Sep 26 01:44:40 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4253 (Wed Sep 26 01:44:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32803192", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false} lago.ssh: DEBUG: start task:81d0c149-6b50-4adf-9d35-f01651cd9f28:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:81d0c149-6b50-4adf-9d35-f01651cd9f28:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running c85eb2ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command c85eb2ac on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command c85eb2ac on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4266 (Wed Sep 26 01:44:54 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4267 (Wed Sep 26 01:44:55 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "cabf6a29", "local_conf_timestamp": 4267, "host-ts": 4266}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4263 (Wed Sep 26 01:44:51 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4263 (Wed Sep 26 01:44:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9de949ea", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:941c8579-9b8f-4691-a6d8-e6fa23801812:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:941c8579-9b8f-4691-a6d8-e6fa23801812:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 966dd6ec on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 966dd6ec on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 966dd6ec on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4174 (Wed Sep 26 01:43:22 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4174 (Wed Sep 26 01:43:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef344a34", "local_conf_timestamp": 4174, "host-ts": 4174}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Wed Sep 26 01:43:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Wed Sep 26 01:43:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cc99e1ab", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d7a66b41-a522-48f5-8a1f-8e8fb9f32468:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d7a66b41-a522-48f5-8a1f-8e8fb9f32468:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 977b6b26 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 977b6b26 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:d6538d87-fdca-41a0-88fd-4f09820ccbfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d6538d87-fdca-41a0-88fd-4f09820ccbfe:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 98e96774 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 98e96774 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 98e96774 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4184 (Wed Sep 26 01:43:32 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4184 (Wed Sep 26 01:43:32 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c21c9ef7", "local_conf_timestamp": 4184, "host-ts": 4184}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4182 (Wed Sep 26 01:43:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4182 (Wed Sep 26 01:43:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cc99e1ab", "local_conf_timestamp": 4182, "host-ts": 4182}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:08a16e5f-0a15-455e-857b-a7420babf805:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:08a16e5f-0a15-455e-857b-a7420babf805:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a022baa4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a022baa4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a022baa4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Wed Sep 26 01:43:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Wed Sep 26 01:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Wed Sep 26 01:43:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4203 (Wed Sep 26 01:43:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:1a8bffad-bd7d-41cc-8bc8-bd7b3d551b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:1a8bffad-bd7d-41cc-8bc8-bd7b3d551b2f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a3170d28 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a3170d28 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a3170d28 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4205 (Wed Sep 26 01:43:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4205 (Wed Sep 26 01:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d8b64cf", "local_conf_timestamp": 4205, "host-ts": 4205}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4203 (Wed Sep 26 01:43:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4203 (Wed Sep 26 01:43:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "94f9ea0d", "local_conf_timestamp": 4203, "host-ts": 4203}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:392c5ef3-b03b-49c3-91a9-195963d2b626:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:392c5ef3-b03b-49c3-91a9-195963d2b626:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a67cd42a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a67cd42a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a67cd42a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Wed Sep 26 01:44:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Wed Sep 26 01:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Wed Sep 26 01:44:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4213 (Wed Sep 26 01:44:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "90bf1508", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:95603aa2-ecc9-48ed-ae43-8061fc3135c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:95603aa2-ecc9-48ed-ae43-8061fc3135c1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a98e6ec6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a98e6ec6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a98e6ec6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Wed Sep 26 01:44:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4216 (Wed Sep 26 01:44:04 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0ab10bb7", "local_conf_timestamp": 4216, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4213 (Wed Sep 26 01:44:00 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4213 (Wed Sep 26 01:44:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "90bf1508", "local_conf_timestamp": 4213, "host-ts": 4213}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:66b9b6bb-1c79-4dda-bc85-906e55c3c37a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:66b9b6bb-1c79-4dda-bc85-906e55c3c37a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ac879efe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ac879efe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ac879efe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Wed Sep 26 01:44:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Wed Sep 26 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Wed Sep 26 01:44:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4223 (Wed Sep 26 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:22298ee9-d865-4828-9cf7-ff814c857a68:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:22298ee9-d865-4828-9cf7-ff814c857a68:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running afcd47d0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command afcd47d0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command afcd47d0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4225 (Wed Sep 26 01:44:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4225 (Wed Sep 26 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba9c1f0a", "local_conf_timestamp": 4225, "host-ts": 4225}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4223 (Wed Sep 26 01:44:11 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4223 (Wed Sep 26 01:44:11 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9c741407", "local_conf_timestamp": 4223, "host-ts": 4223}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:795c2269-a0b6-4929-a1a3-5d7b5d594a0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:795c2269-a0b6-4929-a1a3-5d7b5d594a0f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b2e4baf2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b2e4baf2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b2e4baf2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Wed Sep 26 01:44:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Wed Sep 26 01:44:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Wed Sep 26 01:44:21 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4233 (Wed Sep 26 01:44:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7d3fba27-a87e-4f7e-8387-0abb7f959531:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7d3fba27-a87e-4f7e-8387-0abb7f959531:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b5fcdfbc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b5fcdfbc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b5fcdfbc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4235 (Wed Sep 26 01:44:24 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4236 (Wed Sep 26 01:44:24 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "8da67072", "local_conf_timestamp": 4236, "host-ts": 4235}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4233 (Wed Sep 26 01:44:21 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4233 (Wed Sep 26 01:44:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2b16338c", "local_conf_timestamp": 4233, "host-ts": 4233}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:d2a7fced-d909-40a6-9f7f-0d04cd7dc85d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:d2a7fced-d909-40a6-9f7f-0d04cd7dc85d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b972878c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b972878c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b972878c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Wed Sep 26 01:44:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4246 (Wed Sep 26 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "15580a7d", "local_conf_timestamp": 4246, "host-ts": 4246}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Wed Sep 26 01:44:31 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4244 (Wed Sep 26 01:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a8a25b34", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:04fcaf5e-6061-47d1-8e5f-f189a1a2ba80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:04fcaf5e-6061-47d1-8e5f-f189a1a2ba80:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running bce6191a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command bce6191a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command bce6191a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4246 (Wed Sep 26 01:44:34 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4246 (Wed Sep 26 01:44:34 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "15580a7d", "local_conf_timestamp": 4246, "host-ts": 4246}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4243 (Wed Sep 26 01:44:31 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4244 (Wed Sep 26 01:44:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a8a25b34", "local_conf_timestamp": 4244, "host-ts": 4243}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f780d778-0bd0-4148-a9fb-484d8aa7e20b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f780d778-0bd0-4148-a9fb-484d8aa7e20b:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c032e47c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c032e47c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c032e47c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Wed Sep 26 01:44:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4256 (Wed Sep 26 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b158234b", "local_conf_timestamp": 4256, "host-ts": 4256}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Wed Sep 26 01:44:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4253 (Wed Sep 26 01:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32803192", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:fba632ca-9225-448b-a3d4-97e1026e0168:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:fba632ca-9225-448b-a3d4-97e1026e0168:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c14b6d8e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c14b6d8e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c14b6d8e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4256 (Wed Sep 26 01:44:44 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4256 (Wed Sep 26 01:44:44 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "b158234b", "local_conf_timestamp": 4256, "host-ts": 4256}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4253 (Wed Sep 26 01:44:40 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4253 (Wed Sep 26 01:44:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "32803192", "local_conf_timestamp": 4253, "host-ts": 4253}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:81d0c149-6b50-4adf-9d35-f01651cd9f28:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:81d0c149-6b50-4adf-9d35-f01651cd9f28:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running c85eb2ac on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command c85eb2ac on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command c85eb2ac on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4266 (Wed Sep 26 01:44:54 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4267 (Wed Sep 26 01:44:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "cabf6a29", "local_conf_timestamp": 4267, "host-ts": 4266}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4263 (Wed Sep 26 01:44:51 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4263 (Wed Sep 26 01:44:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9de949ea", "local_conf_timestamp": 4263, "host-ts": 4263}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'