[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 327 - Still Failing!

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/327/ Build Number: 327 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file Changes for Build #322 [Milan Zamazal] Update test-vm.ovf file Changes for Build #323 [Milan Zamazal] Update test-vm.ovf file Changes for Build #324 [Milan Zamazal] Update test-vm.ovf file Changes for Build #325 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #326 [Sahina Bose] hc-suite: Fix version of ansible used Changes for Build #327 [Sahina Bose] hc-suite: Fix version of ansible used ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 8909c558 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "35612a16", "local_conf_timestamp": 4156, "host-ts": 4156}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false} lago.ssh: DEBUG: start task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a198168 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 8a198168 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4167 (Thu Sep 20 01:43:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4167 (Thu Sep 20 01:43:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "189b4b14", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4166 (Thu Sep 20 01:43:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4166 (Thu Sep 20 01:43:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "edcf5346", "local_conf_timestamp": 4166, "host-ts": 4166}, "global_maintenance": false} lago.ssh: DEBUG: start task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9302328e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false} lago.ssh: DEBUG: start task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 970c7d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false} lago.ssh: DEBUG: start task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9a10bc30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4187 (Thu Sep 20 01:43:37 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4187 (Thu Sep 20 01:43:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cdd0276f", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4186 (Thu Sep 20 01:43:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4186 (Thu Sep 20 01:43:36 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false} lago.ssh: DEBUG: start task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 9d8302f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} lago.ssh: DEBUG: start task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a09033b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} lago.ssh: DEBUG: start task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a389cf0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running a732a40a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false} lago.ssh: DEBUG: start task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running aa7d047a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running ad708288 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false} lago.ssh: DEBUG: start task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b074c5c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b189aa8e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false} lago.ssh: DEBUG: start task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running b89697e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4237 (Thu Sep 20 01:44:27 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4237 (Thu Sep 20 01:44:27 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "055173ed", "local_conf_timestamp": 4237, "host-ts": 4237}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Thu Sep 20 01:44:25 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4236 (Thu Sep 20 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f183ffe", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:36ca34cf-d416-4b0e-8b76-6336b4dbcf1f:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 8909c558 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 8909c558 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "35612a16", "local_conf_timestamp": 4156, "host-ts": 4156}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4156 (Thu Sep 20 01:43:06 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4156 (Thu Sep 20 01:43:06 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e1045249", "local_conf_timestamp": 4156, "host-ts": 4156}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:af2ca3b5-50c5-431e-b600-4227511724e6:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a198168 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 8a198168 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:41490c5d-836c-4c8b-976a-613fd76d0307:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8b3e8c14 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4167 (Thu Sep 20 01:43:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4167 (Thu Sep 20 01:43:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "189b4b14", "local_conf_timestamp": 4167, "host-ts": 4167}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4166 (Thu Sep 20 01:43:15 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4166 (Thu Sep 20 01:43:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "edcf5346", "local_conf_timestamp": 4166, "host-ts": 4166}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7a435df0-85fc-4721-8423-d20fc26452d1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9302328e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9302328e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:71a80a58-941b-4ab1-849c-a025e7167cb7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 970c7d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 970c7d3a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4177 (Thu Sep 20 01:43:27 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4177 (Thu Sep 20 01:43:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "40657674", "local_conf_timestamp": 4177, "host-ts": 4177}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4176 (Thu Sep 20 01:43:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4176 (Thu Sep 20 01:43:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e989ac43", "local_conf_timestamp": 4176, "host-ts": 4176}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4cc66cc2-2466-4af4-8319-8a9a03b96bc1:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9a10bc30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9a10bc30 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4187 (Thu Sep 20 01:43:37 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4187 (Thu Sep 20 01:43:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "cdd0276f", "local_conf_timestamp": 4187, "host-ts": 4187}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4186 (Thu Sep 20 01:43:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4186 (Thu Sep 20 01:43:36 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a1539ce8", "local_conf_timestamp": 4186, "host-ts": 4186}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:03a59177-b25a-49c6-b5a6-67e1e5e50703:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9d8302f6 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9d8302f6 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:13fb81dd-76e8-4d80-a6dc-49dee9a3ecd2:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a09033b0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a09033b0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4197 (Thu Sep 20 01:43:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4198 (Thu Sep 20 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f35c043c", "local_conf_timestamp": 4198, "host-ts": 4197}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Thu Sep 20 01:43:46 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Thu Sep 20 01:43:46 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a51563ed", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:69bff1b4-961f-4fc3-9827-8822392ca3a7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a389cf0e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a389cf0e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b1c97c31-aa37-455d-835b-4cf130d18d2d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running a732a40a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command a732a40a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4207 (Thu Sep 20 01:43:57 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4207 (Thu Sep 20 01:43:57 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "2253735e", "local_conf_timestamp": 4207, "host-ts": 4207}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4206 (Thu Sep 20 01:43:55 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4206 (Thu Sep 20 01:43:55 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a0f36ade", "local_conf_timestamp": 4206, "host-ts": 4206}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ea68373c-d41e-4396-9a5c-030b3cff59a9:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running aa7d047a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command aa7d047a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:de21338e-4f32-4f19-afa5-a2c156bda0f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running ad708288 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command ad708288 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4217 (Thu Sep 20 01:44:07 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4218 (Thu Sep 20 01:44:07 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "1cdf500d", "local_conf_timestamp": 4218, "host-ts": 4217}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4216 (Thu Sep 20 01:44:05 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4216 (Thu Sep 20 01:44:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a4b595db", "local_conf_timestamp": 4216, "host-ts": 4216}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7b5e1c7c-a193-4bcf-9858-3e03cc8e54e4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b074c5c0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b074c5c0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a3dc1104-18f2-40fb-a335-3817d9e0f6de:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b189aa8e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b189aa8e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4227 (Thu Sep 20 01:44:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4227 (Thu Sep 20 01:44:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "7bf0462d", "local_conf_timestamp": 4227, "host-ts": 4227}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4226 (Thu Sep 20 01:44:16 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4226 (Thu Sep 20 01:44:16 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "a87e94d4", "local_conf_timestamp": 4226, "host-ts": 4226}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3cfdb007-0047-40de-bad8-3ff228c8a897:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running b89697e2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command b89697e2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4237 (Thu Sep 20 01:44:27 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4237 (Thu Sep 20 01:44:27 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "055173ed", "local_conf_timestamp": 4237, "host-ts": 4237}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Thu Sep 20 01:44:25 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4236 (Thu Sep 20 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9f183ffe", "local_conf_timestamp": 4236, "host-ts": 4236}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'