Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/321/ Build Number: 321 Build Status: Still Failing Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #299 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs Changes for Build #300 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #301 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file Changes for Build #302 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Sandro Bonazzola] ovirt-iso-uploader: branched for 4.2 Changes for Build #303 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #304 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config Changes for Build #305 [Miguel Duarte Barroso] Repos: add python2-ovsdbapp to the reposync config [Gal Ben Haim] Add jobs for ovs-cni Changes for Build #306 [Ales Musil] network: syncutil: Improve sync utility with kwargs Changes for Build #307 [Your Name] network: syncutil: Adding doctext Changes for Build #308 [Milan Zamazal] Update test-vm.ovf file Changes for Build #309 [Milan Zamazal] Update test-vm.ovf file Changes for Build #310 [Milan Zamazal] Update test-vm.ovf file Changes for Build #311 [Milan Zamazal] Update test-vm.ovf file Changes for Build #312 [Milan Zamazal] Update test-vm.ovf file Changes for Build #313 [Milan Zamazal] Update test-vm.ovf file Changes for Build #314 [Milan Zamazal] Update test-vm.ovf file Changes for Build #315 [Milan Zamazal] Update test-vm.ovf file Changes for Build #316 [Milan Zamazal] Update test-vm.ovf file [Ehud Yonasi] standard_pipelines: add nmstate project Changes for Build #317 [Milan Zamazal] Update test-vm.ovf file Changes for Build #318 [Milan Zamazal] Update test-vm.ovf file Changes for Build #319 [Milan Zamazal] Update test-vm.ovf file Changes for Build #320 [Milan Zamazal] Update test-vm.ovf file Changes for Build #321 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 5fd253d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false} lago.ssh: DEBUG: start task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 60dd7c52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 60dd7c52 on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 61fe1d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false} lago.ssh: DEBUG: start task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6a9e0004 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6d936038 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false} lago.ssh: DEBUG: start task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 708dde08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 73876c82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false} lago.ssh: DEBUG: start task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 769e011a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 799ec070 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false} lago.ssh: DEBUG: start task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7ca252a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false} lago.ssh: DEBUG: start task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7fe373fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false} lago.ssh: DEBUG: start task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 834fc6fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false} lago.ssh: DEBUG: start task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 86917a20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false} lago.ssh: DEBUG: start task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 89e318d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4288 (Tue Sep 18 01:45:33 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4288 (Tue Sep 18 01:45:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f547c6f", "local_conf_timestamp": 4288, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4280 (Tue Sep 18 01:45:25 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4280 (Tue Sep 18 01:45:25 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78f0fd60", "local_conf_timestamp": 4280, "host-ts": 4280}, "global_maintenance": false} lago.ssh: DEBUG: start task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 8af57904 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 40, in _wait_for_engine_migration lambda: _check_migration_state(host, state) is False File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 40, in <lambda> lambda: _check_migration_state(host, state) is False File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 103, in _check_migration_state status = _get_he_status(host) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 92, in _get_he_status ret = host.ssh(["hosted-engine", "--vm-status", "--json"]) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 529, in ssh password=self._spec.get('ssh-password'), File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 69, in ssh 'stderr': None File "/usr/lib/python2.7/site-packages/lago/ssh.py", line 247, in drain_ssh_channel 0.1, File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:0125e8af-78f7-4023-b823-e22223815703:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 5fd253d2 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 5fd253d2 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c76c9588-3420-4dfc-99ad-079e9566a222:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 60dd7c52 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 60dd7c52 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ac0198b0-c6c4-4210-ad35-817e8e6eb70a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 61fe1d3a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 61fe1d3a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4215 (Tue Sep 18 01:44:20 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4215 (Tue Sep 18 01:44:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa06e89d", "local_conf_timestamp": 4215, "host-ts": 4215}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4220 (Tue Sep 18 01:44:24 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4220 (Tue Sep 18 01:44:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "96531654", "local_conf_timestamp": 4220, "host-ts": 4220}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:45d5eb99-1019-46ad-8175-9f7013fbe1f3:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6a9e0004 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6a9e0004 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0065fcf4-ebbd-4c89-a278-60bd7ecadc03:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6d936038 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6d936038 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4236 (Tue Sep 18 01:44:41 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4238 (Tue Sep 18 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "0e9b93b6", "local_conf_timestamp": 4238, "host-ts": 4236}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4230 (Tue Sep 18 01:44:35 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4230 (Tue Sep 18 01:44:35 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "9215e951", "local_conf_timestamp": 4230, "host-ts": 4230}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7095f7f7-fc47-46c2-963a-ed0036557d5f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 708dde08 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 708dde08 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1791dc5-b1a4-48d8-84de-e20b790b9746:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 73876c82 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 73876c82 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4248 (Tue Sep 18 01:44:53 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4249 (Tue Sep 18 01:44:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "3d575c1c", "local_conf_timestamp": 4249, "host-ts": 4248}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4240 (Tue Sep 18 01:44:45 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4241 (Tue Sep 18 01:44:45 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8fff1144", "local_conf_timestamp": 4241, "host-ts": 4240}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:24c82ae4-fd1a-411f-b8e5-87204ce63486:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 769e011a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 769e011a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8906c882-b7cd-4bc4-b019-55b667baf207:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 799ec070 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 799ec070 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4258 (Tue Sep 18 01:45:03 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4258 (Tue Sep 18 01:45:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e4635248", "local_conf_timestamp": 4258, "host-ts": 4258}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4250 (Tue Sep 18 01:44:54 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4250 (Tue Sep 18 01:44:54 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fcbeded7", "local_conf_timestamp": 4250, "host-ts": 4250}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b6f57683-5118-40ba-8c2f-4e5f0c82c688:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7ca252a0 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7ca252a0 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0b06b437-0e50-4629-9272-7601d7b33cdc:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7fe373fe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7fe373fe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4268 (Tue Sep 18 01:45:13 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4269 (Tue Sep 18 01:45:14 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "ba4027d9", "local_conf_timestamp": 4269, "host-ts": 4268}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4260 (Tue Sep 18 01:45:04 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4260 (Tue Sep 18 01:45:05 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f075dfd8", "local_conf_timestamp": 4260, "host-ts": 4260}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4ac20e5c-1d7f-4790-8174-a43cfe879caf:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 834fc6fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 834fc6fa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c1bcef6e-b90d-4bf7-b72d-89f3a989cd12:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 86917a20 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 86917a20 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4278 (Tue Sep 18 01:45:23 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4278 (Tue Sep 18 01:45:23 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6374298d", "local_conf_timestamp": 4278, "host-ts": 4278}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4270 (Tue Sep 18 01:45:15 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4270 (Tue Sep 18 01:45:15 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4717f853", "local_conf_timestamp": 4270, "host-ts": 4270}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:be71480c-431e-4f14-837c-93266b5b0113:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 89e318d2 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 89e318d2 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4288 (Tue Sep 18 01:45:33 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4288 (Tue Sep 18 01:45:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "4f547c6f", "local_conf_timestamp": 4288, "host-ts": 4288}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4280 (Tue Sep 18 01:45:25 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4280 (Tue Sep 18 01:45:25 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "78f0fd60", "local_conf_timestamp": 4280, "host-ts": 4280}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:aad5ed24-5d35-4128-bdca-6cb12f8e856d:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8af57904 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'