- Infra - oVirt List Archives

[CQ]: 94228, 1 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 10 Sep '18

10 Sep '18

A system test invoked by the "ovirt-master" change queue including change 94228,1 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/94228/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10176/

2 1

[JIRA] (OVIRT-2483) review the use of temp python-ovsdbapp repo
by Dafna Ron (oVirt JIRA) 10 Sep '18

10 Sep '18

Dafna Ron created OVIRT-2483: -------------------------------- Summary: review the use of temp python-ovsdbapp repo Key: OVIRT-2483 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2483 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Dafna Ron Assignee: infra due to a bug in openstack, package python-ovsdbapp which is a dependency of ovirt-provider-ovn is broken. to fix the issue, [~mdbarroso(a)redhat.com] has created a tmp repo that would replace this specific package until its fixed by openstack. Since this should be a temporary solution and creating a new project that is needed to be maintained by ovirt until openstack will merge the fix may be a too elaborate solution I agreed to add a tmp repo for this package but to review this again in a month to see if there is advancement from openstack side. here is the patch from Miguel: https://gerrit.ovirt.org/#/c/93983/ -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build # 285 - Failure!
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/285/ Build Number: 285 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #285 [Gal Ben Haim] Adding dr suite [Barak Korren] Remove the populate_mock function from mock_runner [Daniel Belenky] mock_runner: store shell cmd in a variable [Barak Korren] mock_runner: Added timeout param [Barak Korren] Make whitelist repo configurable via env vars [Daniel Belenky] stdci_runner: let mock_runner manage timeout [Greg Sheremeta] remove 4.1 change queue from ovirt-engine-nodejs config ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 010_local_maintenance_cli.local_maintenance Error Message: 143 -------------------- >> begin captured logging << -------------------- root: INFO: * Waiting For System Stability... lago.ssh: DEBUG: start task:c3e22bc1-2785-476e-8ba3-ad975b3823a0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: end task:c3e22bc1-2785-476e-8ba3-ad975b3823a0:Get ssh client for lago-he-node-ng-suite-4-2-host-1: lago.ssh: DEBUG: Running 4cc65a9e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4cc65a9e on lago-he-node-ng-suite-4-2-host-1 returned with 0 lago.ssh: DEBUG: Command 4cc65a9e on lago-he-node-ng-suite-4-2-host-1 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4109 (Thu Sep 6 01:43:37 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4109 (Thu Sep 6 01:43:37 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "044228a8", "local_conf_timestamp": 4109, "host-ts": 4109}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4115 (Thu Sep 6 01:43:42 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4115 (Thu Sep 6 01:43:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d059996", "local_conf_timestamp": 4115, "host-ts": 4115}, "global_maintenance": false} lago.ssh: DEBUG: start task:7f537510-6e18-4173-b0f5-d5f6a1c6e460:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:7f537510-6e18-4173-b0f5-d5f6a1c6e460:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4dde7bbe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local lago.ssh: DEBUG: Command 4dde7bbe on lago-he-node-ng-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for engine to migrate... lago.ssh: DEBUG: start task:94120632-275d-478f-9cac-0ff1eb6c2e63:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:94120632-275d-478f-9cac-0ff1eb6c2e63:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 4ef81abe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 4ef81abe on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 4ef81abe on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4119 (Thu Sep 6 01:43:47 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=4120 (Thu Sep 6 01:43:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8ad483f0", "local_conf_timestamp": 4120, "host-ts": 4119}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4115 (Thu Sep 6 01:43:42 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4115 (Thu Sep 6 01:43:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d059996", "local_conf_timestamp": 4115, "host-ts": 4115}, "global_maintenance": false} lago.ssh: DEBUG: start task:82d6fe01-3354-4477-9210-df6ac2d4d420:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:82d6fe01-3354-4477-9210-df6ac2d4d420:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5626eb30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5626eb30 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5626eb30 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4131 (Thu Sep 6 01:43:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4130 (Thu Sep 6 01:43:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "65ca1ba5", "local_conf_timestamp": 4130, "host-ts": 4131}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4125 (Thu Sep 6 01:43:53 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4126 (Thu Sep 6 01:43:53 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18c7c923", "local_conf_timestamp": 4126, "host-ts": 4125}, "global_maintenance": false} lago.ssh: DEBUG: start task:b3211ae7-4af2-469b-92c0-38ab6f07c5c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:b3211ae7-4af2-469b-92c0-38ab6f07c5c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 59979f1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 59979f1c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 59979f1c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4141 (Thu Sep 6 01:44:09 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4141 (Thu Sep 6 01:44:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "49290bc0", "local_conf_timestamp": 4141, "host-ts": 4141}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4135 (Thu Sep 6 01:44:02 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4135 (Thu Sep 6 01:44:02 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8588679c", "local_conf_timestamp": 4135, "host-ts": 4135}, "global_maintenance": false} lago.ssh: DEBUG: start task:f62305f7-c324-4ee2-ba11-d092ac823600:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f62305f7-c324-4ee2-ba11-d092ac823600:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 5dfd4692 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5dfd4692 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5dfd4692 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4151 (Thu Sep 6 01:44:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4151 (Thu Sep 6 01:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e71a3502", "local_conf_timestamp": 4151, "host-ts": 4151}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4145 (Thu Sep 6 01:44:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4145 (Thu Sep 6 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false} lago.ssh: DEBUG: start task:a553c343-1311-45f3-9fc3-ed53d93b21d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a553c343-1311-45f3-9fc3-ed53d93b21d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 61aded00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 61aded00 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 61aded00 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4151 (Thu Sep 6 01:44:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4151 (Thu Sep 6 01:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e71a3502", "local_conf_timestamp": 4151, "host-ts": 4151}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4145 (Thu Sep 6 01:44:13 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4145 (Thu Sep 6 01:44:13 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false} lago.ssh: DEBUG: start task:bb63e900-36b1-47d5-8bd7-b6b67eed6f7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:bb63e900-36b1-47d5-8bd7-b6b67eed6f7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 64b1136a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 64b1136a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 64b1136a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4151 (Thu Sep 6 01:44:18 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4151 (Thu Sep 6 01:44:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e71a3502", "local_conf_timestamp": 4151, "host-ts": 4151}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4155 (Thu Sep 6 01:44:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4155 (Thu Sep 6 01:44:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eb23501a", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false} lago.ssh: DEBUG: start task:97447141-6b3c-4188-ad5b-a8bbe1cc45db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:97447141-6b3c-4188-ad5b-a8bbe1cc45db:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 67a5dc18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 67a5dc18 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 67a5dc18 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4161 (Thu Sep 6 01:44:29 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4162 (Thu Sep 6 01:44:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "573721bf", "local_conf_timestamp": 4162, "host-ts": 4161}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4155 (Thu Sep 6 01:44:22 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4155 (Thu Sep 6 01:44:22 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eb23501a", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false} lago.ssh: DEBUG: start task:dcbacc88-4c96-4575-a53a-31b073b1e31e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:dcbacc88-4c96-4575-a53a-31b073b1e31e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6ada63f4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6ada63f4 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6ada63f4 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4171 (Thu Sep 6 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4172 (Thu Sep 6 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f9041f7d", "local_conf_timestamp": 4172, "host-ts": 4171}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Thu Sep 6 01:44:32 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4165 (Thu Sep 6 01:44:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false} lago.ssh: DEBUG: start task:90045a52-c293-4837-bfef-33a5a1910dc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:90045a52-c293-4837-bfef-33a5a1910dc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 6df25e2a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 6df25e2a on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 6df25e2a on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4171 (Thu Sep 6 01:44:39 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4172 (Thu Sep 6 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f9041f7d", "local_conf_timestamp": 4172, "host-ts": 4171}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4165 (Thu Sep 6 01:44:32 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4165 (Thu Sep 6 01:44:33 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false} lago.ssh: DEBUG: start task:a7476663-b52b-4275-8cc2-cd9ff3b88a3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:a7476663-b52b-4275-8cc2-cd9ff3b88a3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 71077262 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 71077262 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 71077262 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Thu Sep 6 01:44:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4181 (Thu Sep 6 01:44:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "edb81fdc", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Thu Sep 6 01:44:43 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4176 (Thu Sep 6 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9832724", "local_conf_timestamp": 4176, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:42a25e42-c90a-456e-8669-82df79c7507c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:42a25e42-c90a-456e-8669-82df79c7507c:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 74595c8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 74595c8c on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 74595c8c on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4181 (Thu Sep 6 01:44:48 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4181 (Thu Sep 6 01:44:49 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "edb81fdc", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4175 (Thu Sep 6 01:44:43 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4176 (Thu Sep 6 01:44:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9832724", "local_conf_timestamp": 4176, "host-ts": 4175}, "global_maintenance": false} lago.ssh: DEBUG: start task:22a966cf-52a3-43b7-98b0-a28f0df8f67e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:22a966cf-52a3-43b7-98b0-a28f0df8f67e:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 77bc4bdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 77bc4bdc on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 77bc4bdc on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Thu Sep 6 01:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4192 (Thu Sep 6 01:45:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "29ba0d11", "local_conf_timestamp": 4192, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Thu Sep 6 01:44:52 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4185 (Thu Sep 6 01:44:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6f6d73ad", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:f9fa6fdb-2c5a-41b9-ac58-6ec7a8e8ef3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:f9fa6fdb-2c5a-41b9-ac58-6ec7a8e8ef3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 78d063fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 78d063fa on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 78d063fa on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4191 (Thu Sep 6 01:44:59 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4192 (Thu Sep 6 01:45:00 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "29ba0d11", "local_conf_timestamp": 4192, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4185 (Thu Sep 6 01:44:52 2018)\nhost-id=2\nscore=3000\nvm_conf_refresh_time=4185 (Thu Sep 6 01:44:52 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6f6d73ad", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false} lago.ssh: DEBUG: start task:095cd3ad-d465-46bb-8cd1-fd07a1a124ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: end task:095cd3ad-d465-46bb-8cd1-fd07a1a124ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0: lago.ssh: DEBUG: Running 7fdb3a94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 7fdb3a94 on lago-he-node-ng-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 7fdb3a94 on lago-he-node-ng-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4202 (Thu Sep 6 01:45:10 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=4203 (Thu Sep 6 01:45:10 2018)\nconf_on_shared_storage=True\nmaintenance=True\nstate=LocalMaintenance\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8a558591", "local_conf_timestamp": 4203, "host-ts": 4202}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4196 (Thu Sep 6 01:45:03 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4196 (Thu Sep 6 01:45:03 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a9977fba", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false} root: INFO: * Engine has migrated. root: INFO: * Waiting For System Stability... cli: DEBUG: signal 15 was caught --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance _wait_for_engine_migration(host, he_index, "bad", "Migration Destination") File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 45, in _wait_for_engine_migration time.sleep(wait_value) File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler sys.exit(128 + signum) '143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:c3e22bc1-2785-476e-8ba3-ad975b3823a0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:c3e22bc1-2785-476e-8ba3-ad975b3823a0:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 4cc65a9e on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4cc65a9e on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 4cc65a9e on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4109 (Thu Sep 6 01:43:37 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4109 (Thu Sep 6 01:43:37 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "044228a8", "local_conf_timestamp": 4109, "host-ts": 4109}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4115 (Thu Sep 6 01:43:42 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4115 (Thu Sep 6 01:43:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d059996", "local_conf_timestamp": 4115, "host-ts": 4115}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:7f537510-6e18-4173-b0f5-d5f6a1c6e460:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7f537510-6e18-4173-b0f5-d5f6a1c6e460:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4dde7bbe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 4dde7bbe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:94120632-275d-478f-9cac-0ff1eb6c2e63:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:94120632-275d-478f-9cac-0ff1eb6c2e63:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 4ef81abe on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 4ef81abe on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 4ef81abe on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4119 (Thu Sep 6 01:43:47 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=4120 (Thu Sep 6 01:43:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8ad483f0", "local_conf_timestamp": 4120, "host-ts": 4119}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4115 (Thu Sep 6 01:43:42 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4115 (Thu Sep 6 01:43:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8d059996", "local_conf_timestamp": 4115, "host-ts": 4115}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:82d6fe01-3354-4477-9210-df6ac2d4d420:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:82d6fe01-3354-4477-9210-df6ac2d4d420:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5626eb30 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5626eb30 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5626eb30 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4131 (Thu Sep 6 01:43:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4130 (Thu Sep 6 01:43:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "65ca1ba5", "local_conf_timestamp": 4130, "host-ts": 4131}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4125 (Thu Sep 6 01:43:53 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4126 (Thu Sep 6 01:43:53 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "18c7c923", "local_conf_timestamp": 4126, "host-ts": 4125}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:b3211ae7-4af2-469b-92c0-38ab6f07c5c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:b3211ae7-4af2-469b-92c0-38ab6f07c5c4:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 59979f1c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 59979f1c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 59979f1c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4141 (Thu Sep 6 01:44:09 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4141 (Thu Sep 6 01:44:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "49290bc0", "local_conf_timestamp": 4141, "host-ts": 4141}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4135 (Thu Sep 6 01:44:02 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4135 (Thu Sep 6 01:44:02 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8588679c", "local_conf_timestamp": 4135, "host-ts": 4135}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f62305f7-c324-4ee2-ba11-d092ac823600:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f62305f7-c324-4ee2-ba11-d092ac823600:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5dfd4692 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5dfd4692 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5dfd4692 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4151 (Thu Sep 6 01:44:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4151 (Thu Sep 6 01:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e71a3502", "local_conf_timestamp": 4151, "host-ts": 4151}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4145 (Thu Sep 6 01:44:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4145 (Thu Sep 6 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a553c343-1311-45f3-9fc3-ed53d93b21d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a553c343-1311-45f3-9fc3-ed53d93b21d5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 61aded00 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 61aded00 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 61aded00 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4151 (Thu Sep 6 01:44:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4151 (Thu Sep 6 01:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e71a3502", "local_conf_timestamp": 4151, "host-ts": 4151}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4145 (Thu Sep 6 01:44:13 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4145 (Thu Sep 6 01:44:13 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ef65af1f", "local_conf_timestamp": 4145, "host-ts": 4145}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:bb63e900-36b1-47d5-8bd7-b6b67eed6f7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:bb63e900-36b1-47d5-8bd7-b6b67eed6f7e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 64b1136a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 64b1136a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 64b1136a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4151 (Thu Sep 6 01:44:18 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4151 (Thu Sep 6 01:44:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "e71a3502", "local_conf_timestamp": 4151, "host-ts": 4151}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4155 (Thu Sep 6 01:44:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4155 (Thu Sep 6 01:44:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eb23501a", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:97447141-6b3c-4188-ad5b-a8bbe1cc45db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:97447141-6b3c-4188-ad5b-a8bbe1cc45db:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 67a5dc18 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 67a5dc18 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 67a5dc18 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4161 (Thu Sep 6 01:44:29 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4162 (Thu Sep 6 01:44:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "573721bf", "local_conf_timestamp": 4162, "host-ts": 4161}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4155 (Thu Sep 6 01:44:22 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4155 (Thu Sep 6 01:44:22 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "eb23501a", "local_conf_timestamp": 4155, "host-ts": 4155}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dcbacc88-4c96-4575-a53a-31b073b1e31e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dcbacc88-4c96-4575-a53a-31b073b1e31e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6ada63f4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6ada63f4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6ada63f4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4171 (Thu Sep 6 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4172 (Thu Sep 6 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f9041f7d", "local_conf_timestamp": 4172, "host-ts": 4171}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Thu Sep 6 01:44:32 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4165 (Thu Sep 6 01:44:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:90045a52-c293-4837-bfef-33a5a1910dc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:90045a52-c293-4837-bfef-33a5a1910dc5:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6df25e2a on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6df25e2a on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6df25e2a on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4171 (Thu Sep 6 01:44:39 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4172 (Thu Sep 6 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "f9041f7d", "local_conf_timestamp": 4172, "host-ts": 4171}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4165 (Thu Sep 6 01:44:32 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4165 (Thu Sep 6 01:44:33 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "54cc899b", "local_conf_timestamp": 4165, "host-ts": 4165}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a7476663-b52b-4275-8cc2-cd9ff3b88a3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a7476663-b52b-4275-8cc2-cd9ff3b88a3c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71077262 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71077262 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71077262 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Thu Sep 6 01:44:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4181 (Thu Sep 6 01:44:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "edb81fdc", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Thu Sep 6 01:44:43 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4176 (Thu Sep 6 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9832724", "local_conf_timestamp": 4176, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:42a25e42-c90a-456e-8669-82df79c7507c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:42a25e42-c90a-456e-8669-82df79c7507c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 74595c8c on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 74595c8c on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 74595c8c on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4181 (Thu Sep 6 01:44:48 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4181 (Thu Sep 6 01:44:49 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "edb81fdc", "local_conf_timestamp": 4181, "host-ts": 4181}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4175 (Thu Sep 6 01:44:43 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4176 (Thu Sep 6 01:44:43 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "c9832724", "local_conf_timestamp": 4176, "host-ts": 4175}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:22a966cf-52a3-43b7-98b0-a28f0df8f67e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:22a966cf-52a3-43b7-98b0-a28f0df8f67e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 77bc4bdc on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 77bc4bdc on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 77bc4bdc on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Thu Sep 6 01:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4192 (Thu Sep 6 01:45:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "29ba0d11", "local_conf_timestamp": 4192, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Thu Sep 6 01:44:52 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4185 (Thu Sep 6 01:44:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6f6d73ad", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:f9fa6fdb-2c5a-41b9-ac58-6ec7a8e8ef3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f9fa6fdb-2c5a-41b9-ac58-6ec7a8e8ef3f:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 78d063fa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 78d063fa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 78d063fa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4191 (Thu Sep 6 01:44:59 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=4192 (Thu Sep 6 01:45:00 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "VM migrated away successfully", "health": "bad", "vm": "down", "detail": "Down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "29ba0d11", "local_conf_timestamp": 4192, "host-ts": 4191}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4185 (Thu Sep 6 01:44:52 2018)\\nhost-id=2\\nscore=3000\\nvm_conf_refresh_time=4185 (Thu Sep 6 01:44:52 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6f6d73ad", "local_conf_timestamp": 4185, "host-ts": 4185}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:095cd3ad-d465-46bb-8cd1-fd07a1a124ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:095cd3ad-d465-46bb-8cd1-fd07a1a124ed:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7fdb3a94 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7fdb3a94 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7fdb3a94 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4202 (Thu Sep 6 01:45:10 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=4203 (Thu Sep 6 01:45:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=True\\nstate=LocalMaintenance\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 0, "stopped": false, "maintenance": true, "crc32": "8a558591", "local_conf_timestamp": 4203, "host-ts": 4202}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=4196 (Thu Sep 6 01:45:03 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=4196 (Thu Sep 6 01:45:03 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a9977fba", "local_conf_timestamp": 4196, "host-ts": 4196}, "global_maintenance": false}\n\nroot: INFO: * Engine has migrated.\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'

1 13

[CQ]: 94186,2 (vdsm) failed "ovirt-4.2" system tests
by oVirt Jenkins 10 Sep '18

10 Sep '18

Change 94186,2 (vdsm) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94186/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3069/

1 0

[CQ]: 94153,3 (ovirt-node-ng) failed "ovirt-master" system tests
by oVirt Jenkins 10 Sep '18

10 Sep '18

Change 94153,3 (ovirt-node-ng) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94153/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10175/

2 1

[JIRA] (OVIRT-2482) ovirt-node-ng on ovirt-master failure on test verify_suspend_resume_vm0
by Dafna Ron (oVirt JIRA) 10 Sep '18

10 Sep '18

Dafna Ron created OVIRT-2482: -------------------------------- Summary: ovirt-node-ng on ovirt-master failure on test verify_suspend_resume_vm0 Key: OVIRT-2482 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2482 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Dafna Ron Assignee: infra http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10175/ The change reported is not related to the failure. I can see the failure is due to host0 responding with an invalid argument (which many just mean the host is not responding but seems like a reporting exception related to virt team) https://pastebin.com/pm6x0W62 -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[oVirt Jenkins] ovirt-node-ng-image_master_build-artifacts-fc28-x86_64 - Build # 48 - Failure!
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-node-ng-image_master_build-artifacts-fc2… Build: http://jenkins.ovirt.org/job/ovirt-node-ng-image_master_build-artifacts-fc2… Build Number: 48 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #48 [Yuval Turgeman] stdci: move build-artifacts to a v1 job ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] ovirt-node-ng-image_master_build-artifacts-el7-x86_64 - Build # 48 - Failure!
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-node-ng-image_master_build-artifacts-el7… Build: http://jenkins.ovirt.org/job/ovirt-node-ng-image_master_build-artifacts-el7… Build Number: 48 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #48 [Yuval Turgeman] stdci: move build-artifacts to a v1 job ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] ovirt-appliance_master_build-artifacts-fc28-x86_64 - Build # 54 - Failure!
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-fc28-x8… Build: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-fc28-x8… Build Number: 54 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #54 [Yuval Turgeman] automation: remove build-artifacts* from stdci-v2 ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] ovirt-appliance_master_build-artifacts-el7-x86_64 - Build # 912 - Failure!
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el7-x86… Build: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el7-x86… Build Number: 912 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #912 [Yuval Turgeman] automation: remove build-artifacts* from stdci-v2 ----------------- Failed Tests: ----------------- No tests ran.

1 1

[oVirt Jenkins] ovirt-system-tests_compat-4.2-suite-master - Build # 148 - Failure!
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/148/ Build Number: 148 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #148 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_master_storage_domain_42 Error Message: Could not find hosts that are up in DC test-dc -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:3e326c56-0189-415d-9472-726ebc93e100:Get ssh client for lago-compat-4-2-suite-master-engine: lago.ssh: DEBUG: end task:3e326c56-0189-415d-9472-726ebc93e100:Get ssh client for lago-compat-4-2-suite-master-engine: lago.ssh: DEBUG: Running 6ab4ea02 on lago-compat-4-2-suite-master-engine: cat /root/multipath.txt lago.ssh: DEBUG: Command 6ab4ea02 on lago-compat-4-2-suite-master-engine returned with 0 lago.ssh: DEBUG: Command 6ab4ea02 on lago-compat-4-2-suite-master-engine output: 360014053b222f0bab4746eeafda553e1 360014055f95c6e4b14c46458b49cadf1 360014058924d8b99d2240c19b8f5d790 36001405abef02a6719d421582ddfae22 36001405c5f42446c03f4e5582f768b51 --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/002_bootstrap.py", line 402, in add_master_storage_domain add_iscsi_storage_domain(prefix) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/002_bootstrap.py", line 546, in add_iscsi_storage_domain host=_random_host_from_dc(api, DC_NAME), File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/002_bootstrap.py", line 121, in _random_host_from_dc return _hosts_in_dc(api, dc_name, True) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/002_bootstrap.py", line 118, in _hosts_in_dc raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name) 'Could not find hosts that are up in DC test-dc\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:3e326c56-0189-415d-9472-726ebc93e100:Get ssh client for lago-compat-4-2-suite-master-engine:\nlago.ssh: DEBUG: end task:3e326c56-0189-415d-9472-726ebc93e100:Get ssh client for lago-compat-4-2-suite-master-engine:\nlago.ssh: DEBUG: Running 6ab4ea02 on lago-compat-4-2-suite-master-engine: cat /root/multipath.txt\nlago.ssh: DEBUG: Command 6ab4ea02 on lago-compat-4-2-suite-master-engine returned with 0\nlago.ssh: DEBUG: Command 6ab4ea02 on lago-compat-4-2-suite-master-engine output:\n 360014053b222f0bab4746eeafda553e1\n360014055f95c6e4b14c46458b49cadf1\n360014058924d8b99d2240c19b8f5d790\n36001405abef02a6719d421582ddfae22\n36001405c5f42446c03f4e5582f768b51\n\n--------------------- >> end captured logging << ---------------------'

1 1

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #1931
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 Commit message: "stdci_runner.groovy: utilize DSL's timeout cfg" > git rev-list --no-walk d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins2674276755294138368.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7' Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 578
by jenkins＠jenkins.phx.ovirt.org 10 Sep '18

10 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/578//artifact/exported-a… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1846
by jenkins＠jenkins.phx.ovirt.org 09 Sep '18

09 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1846/displ…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 Commit message: "stdci_runner.groovy: utilize DSL's timeout cfg" > git rev-list --no-walk d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins986033277216880469.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 209, in main my.doSackSetup(arches) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 741, in doSackSetup return self._getSacks(archlist=archlist, thisrepo=thisrepo) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks self.repos.populateSack(which=repos) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 386, in populateSack sack.populate(repo, mdtype, callback, cacheonly) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 256, in populate ret = misc.repo_gen_decompress(xml, gen, cached=repo.cache) File "/usr/lib/python2.7/site-packages/yum/misc.py", line 1165, in repo_gen_decompress return decompress(filename, dest=dest, check_timestamps=True) File "/usr/lib/python2.7/site-packages/yum/misc.py", line 1150, in decompress _decompress_chunked(filename, out, ztype) File "/usr/lib/python2.7/site-packages/yum/misc.py", line 791, in _decompress_chunked raise Errors.MiscError, msg yum.Errors.MiscError: Error reading from file /home/jenkins/mirrors_cache/epel-el6/6f3bda12f08799dd2da2b7e6791ce9b04b388a41fbc7dae1be585a2291305854-primary.xml.gz: Not a gzipped file Build step 'Execute shell' marked build as failure

1 1

[oVirt Jenkins] ovirt-node-ng_master_build-artifacts-fc28-x86_64 - Build # 44 - Failure!
by jenkins＠jenkins.phx.ovirt.org 09 Sep '18

09 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-fc28-x86_… Build: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-fc28-x86_… Build Number: 44 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #44 [Evgheni Dereveanchin] Install centos-release-qemu-ev from global_setup [Ehud Yonasi] docker_cleanup: Global var for repo cache [Evgheni Dereveanchin] update release RPM name containing Virt SIG pubkey [Evgheni Dereveanchin] Remove node-filter for HC jobs [Yuval Turgeman] build: add version and release.dist to iso name ----------------- Failed Tests: ----------------- No tests ran.

1 17

[oVirt Jenkins] ovirt-node-ng_master_build-artifacts-el7-x86_64 - Build # 917 - Failure!
by jenkins＠jenkins.phx.ovirt.org 09 Sep '18

09 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build Number: 917 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #917 [Sandro Bonazzola] Bump version to 4.3.0 ----------------- Failed Tests: ----------------- No tests ran.

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1844
by jenkins＠jenkins.phx.ovirt.org 09 Sep '18

09 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1844/displ…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 Commit message: "stdci_runner.groovy: utilize DSL's timeout cfg" > git rev-list --no-walk d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins5316593278641742250.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7' Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 577
by jenkins＠jenkins.phx.ovirt.org 09 Sep '18

09 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/577//artifact/exported-a… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 576
by jenkins＠jenkins.phx.ovirt.org 08 Sep '18

08 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/576//artifact/exported-a… Cheers, Jenkins

1 0

[CQ]: 94223, 2 (ovirt-engine-api-model) failed "ovirt-master" system tests
by oVirt Jenkins 07 Sep '18

07 Sep '18

Change 94223,2 (ovirt-engine-api-model) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94223/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10168/

1 0

[CQ]: 65269d3 (ovirt-ansible-manageiq) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 07 Sep '18

07 Sep '18

A system test invoked by the "ovirt-4.2" change queue including change 65269d3 (ovirt-ansible-manageiq) failed. However, this change seems not to be the root cause for this failure. Change 9e1fdd0 (ovirt-ansible-manageiq) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 9e1fdd0 (ovirt-ansible- manageiq) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-manageiq/commit/65269d3f3e8442828fe8… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-manageiq/commit/9e1fdd087daecbf2b201… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3061/

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 451 - Failure!
by jenkins＠jenkins.phx.ovirt.org 07 Sep '18

07 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/451/ Build Number: 451 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #451 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: Host lago-hc-basic-suite-4-2-host1 is in non operational state -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7fce8965c0c8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 259, in _host_is_up_4 raise RuntimeError('Host %s is in non operational state' % api_host.name) RuntimeError: Host lago-hc-basic-suite-4-2-host1 is in non operational state --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 229, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 273, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 259, in _host_is_up_4 raise RuntimeError('Host %s is in non operational state' % api_host.name) 'Host lago-hc-basic-suite-4-2-host1 is in non operational state\n-------------------- >> begin captured logging << --------------------\novirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7fce8965c0c8>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 259, in _host_is_up_4\n raise RuntimeError(\'Host %s is in non operational state\' % api_host.name)\nRuntimeError: Host lago-hc-basic-suite-4-2-host1 is in non operational state\n--------------------- >> end captured logging << ---------------------'

1 1

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1838
by jenkins＠jenkins.phx.ovirt.org 07 Sep '18

07 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1838/displ…> Changes: [Barak Korren] Enable CI for kubevirt/client-python [Daniel Belenky] Add timeout config to stdci dsl [Daniel Belenky] stdci_runner.groovy: utilize DSL's timeout cfg ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f d24ebbf93915e3b9890cefb36aa04a8dd3a245b8 Commit message: "stdci_runner.groovy: utilize DSL's timeout cfg" > git rev-list --no-walk 3e30982d0c0ff59fd472a6a75efc2201d8c9c2de # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins1400443596078583436.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7' Build step 'Execute shell' marked build as failure

1 1

[oVirt Jenkins] ovirt-system-tests_performance-suite-master - Build # 652 - Failure!
by jenkins＠jenkins.phx.ovirt.org 07 Sep '18

07 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_performance-suite-master/65… Build Number: 652 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #652 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Barak Korren] Enable CI for kubevirt/client-python [Daniel Belenky] Add timeout config to stdci dsl [Daniel Belenky] stdci_runner.groovy: utilize DSL's timeout cfg ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 040_add_hosts_vms.add_master_storage_domain Error Message: Fault reason is "Operation Failed". Fault detail is "[Cannot add storage server connection when Host status is not up]". HTTP response code is 409. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 340, in add_master_storage_domain add_nfs_storage_domain(prefix) File "/home/jenkins/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 344, in add_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, SD_NFS_NAME, SD_NFS_HOST_NAME, SD_NFS_PATH, nfs_version='v4_2') File "/home/jenkins/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 386, in add_generic_nfs_storage_domain_4 _add_storage_domain_4(api, p) File "/home/jenkins/workspace/ovirt-system-tests_performance-suite-master/ovirt-system-tests/performance-suite-master/test-scenarios/040_add_hosts_vms.py", line 314, in _add_storage_domain_4 sd = sds_service.add(p) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 24011, in add return self._internal_add(storage_domain, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 232, in _internal_add return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 229, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault self._raise_error(response, body) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Cannot add storage server connection when Host status is not up]". HTTP response code is 409.

1 1

oVirt infra daily report - unstable production jobs - 575
by jenkins＠jenkins.phx.ovirt.org 07 Sep '18

07 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/575//artifact/exported-a… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2481) Developer permission request
by Emil Natan (oVirt JIRA) 06 Sep '18

06 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2481?page=com.atlassian.jira.… ] Emil Natan reassigned OVIRT-2481: --------------------------------- Assignee: Emil Natan (was: infra) > Developer permission request > ---------------------------- > > Key: OVIRT-2481 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2481 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Ahmad Khiet > Assignee: Emil Natan > > Hi, > I want developer permission on ovirt jenkins, as a new developer working > at redhat, storage team - Tal's team. > user : akhiet > Thanks, > Ahmad -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[JIRA] (OVIRT-2481) Developer permission request
by Ahmad Khiet (oVirt JIRA) 06 Sep '18

06 Sep '18

Ahmad Khiet created OVIRT-2481: ---------------------------------- Summary: Developer permission request Key: OVIRT-2481 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2481 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Ahmad Khiet Assignee: infra Hi, I want developer permission on ovirt jenkins, as a new developer working at redhat, storage team - Tal's team. user : akhiet Thanks, Ahmad -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[CQ]: 93950,18 (ovirt-imageio) failed "ovirt-master" system tests
by oVirt Jenkins 06 Sep '18

06 Sep '18

Change 93950,18 (ovirt-imageio) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/93950/18 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10121/

2 1

[CQ]: 94165, 5 (ovirt-engine-nodejs-modules) failed "ovirt-master" system tests
by oVirt Jenkins 06 Sep '18

06 Sep '18

Change 94165,5 (ovirt-engine-nodejs-modules) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94165/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10157/

2 1

Change in stdci-staging[master]: Check timeout functionality
by review＠gerrit-staging.phx.ovirt.org 06 Sep '18

06 Sep '18

From Daniel Belenky <dbelenky(a)redhat.com>: Daniel Belenky has posted comments on this change. ( https://gerrit-staging.phx.ovirt.org/213 ) Change subject: Check timeout functionality ...................................................................... Patch Set 1: ci test please -- To view, visit https://gerrit-staging.phx.ovirt.org/213 To unsubscribe, visit https://gerrit-staging.phx.ovirt.org/settings Gerrit-Project: stdci-staging Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8264e2efd72cf4c969dd9d2a149a3fbd45f6feb7 Gerrit-Change-Number: 213 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Belenky <dbelenky(a)redhat.com> Gerrit-Reviewer: Daniel Belenky <dbelenky(a)redhat.com> Gerrit-Reviewer: Jenkins CI <infra(a)ovirt.org> Gerrit-Comment-Date: Thu, 06 Sep 2018 12:32:06 +0000 Gerrit-HasComments: No

1 0

[oVirt CI] standard-enqueue - Build #16308 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 06 Sep '18

06 Sep '18

Build: http://jenkins.ovirt.org/job/standard-enqueue/16308/ Build Name: #16308 Build Description: Gerrit: 94165 - ovirt-engine-nodejs-modules (4.1, 4.2, master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/94165 - title: add pre-seed for ovirt-web-ui - project: ovirt-engine-nodejs-modules - branch: master - author: Greg Sheremeta <gshereme(a)redhat.com>

1 0

Change in stdci-staging[master]: Check timeout functionality
by review＠gerrit-staging.phx.ovirt.org 06 Sep '18

06 Sep '18

From Daniel Belenky <dbelenky(a)redhat.com>: Hello Jenkins CI, I'd like you to reexamine a change. Please visit https://gerrit-staging.phx.ovirt.org/213 to look at the new patch set (#2). Change subject: Check timeout functionality ...................................................................... Check timeout functionality Change-Id: I8264e2efd72cf4c969dd9d2a149a3fbd45f6feb7 Signed-off-by: Daniel Belenky <dbelenky(a)redhat.com> --- A automation/check-patch.do-not-timeout.sh A automation/check-patch.do-timeout.sh M stdci.yaml 3 files changed, 19 insertions(+), 3 deletions(-) git pull ssh://gerrit-staging.phx.ovirt.org:29418/stdci-staging refs/changes/13/213/2 -- To view, visit https://gerrit-staging.phx.ovirt.org/213 To unsubscribe, visit https://gerrit-staging.phx.ovirt.org/settings Gerrit-Project: stdci-staging Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8264e2efd72cf4c969dd9d2a149a3fbd45f6feb7 Gerrit-Change-Number: 213 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Belenky <dbelenky(a)redhat.com> Gerrit-Reviewer: Daniel Belenky <dbelenky(a)redhat.com> Gerrit-Reviewer: Jenkins CI <infra(a)ovirt.org>

1 0

Change in stdci-staging[master]: Check timeout functionality
by review＠gerrit-staging.phx.ovirt.org 06 Sep '18

06 Sep '18

From Daniel Belenky <dbelenky(a)redhat.com>: Daniel Belenky has posted comments on this change. ( https://gerrit-staging.phx.ovirt.org/213 ) Change subject: Check timeout functionality ...................................................................... Patch Set 1: ci test please -- To view, visit https://gerrit-staging.phx.ovirt.org/213 To unsubscribe, visit https://gerrit-staging.phx.ovirt.org/settings Gerrit-Project: stdci-staging Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8264e2efd72cf4c969dd9d2a149a3fbd45f6feb7 Gerrit-Change-Number: 213 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Belenky <dbelenky(a)redhat.com> Gerrit-Reviewer: Daniel Belenky <dbelenky(a)redhat.com> Gerrit-Reviewer: Jenkins CI <infra(a)ovirt.org> Gerrit-Comment-Date: Thu, 06 Sep 2018 11:45:31 +0000 Gerrit-HasComments: No

1 0

[JIRA] (OVIRT-2479) CQ Failure - Alot of ovirt-hosted-engine-setup messages.
by Dafna Ron (oVirt JIRA) 06 Sep '18

06 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2479?page=com.atlassian.jira.… ] Dafna Ron updated OVIRT-2479: ----------------------------- Issue Type: Bug (was: Outage) > CQ Failure - Alot of ovirt-hosted-engine-setup messages. > -------------------------------------------------------- > > Key: OVIRT-2479 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2479 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Change Queue, OST > Reporter: Ehud Yonasi > Assignee: infra > Labels: cq-issue-traking, ost_failures > > Since 3rd september there were alot of changes coming out of the blue to CQ about ovirt-hosted-engine-setup. > while investigating why it happened, I found out that the remote repo was deleted somehow and was restored. > That restoration caused all the project patches all-at-once triggers hundreds of jobs in build-art + CQ. > In order to recover from that situation i had to find the last patch of that project and asked for a re-merge (ci re-merge please). > After that change and multiple runs of CQ it passed. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 447 - Failure!
by jenkins＠jenkins.phx.ovirt.org 06 Sep '18

06 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/447/ Build Number: 447 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #447 [Gal Ben Haim] Adding dr suite ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.snapshot_merge Error Message: status: 409 reason: Conflict detail: Cannot remove Snapshot. Snapshot is currently being created for VM vm0. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 69, in wrapper return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 200, in snapshot_merge api.vms.get(VM0_NAME).snapshots.list()[-2].delete() File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 34001, in delete headers={"Correlation-Id":correlation_id,"Content-type":None} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 57, in delete return self.request('DELETE', url, body, headers, cls=cls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 409 reason: Conflict detail: Cannot remove Snapshot. Snapshot is currently being created for VM vm0.

1 2

[oVirt Jenkins] ovirt-system-tests_compat-4.2-suite-master - Build # 137 - Failure!
by jenkins＠jenkins.phx.ovirt.org 06 Sep '18

06 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/137/ Build Number: 137 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #137 [Gal Ben Haim] Adding dr suite [Barak Korren] Remove the populate_mock function from mock_runner [Daniel Belenky] mock_runner: store shell cmd in a variable [Barak Korren] mock_runner: Added timeout param [Barak Korren] Make whitelist repo configurable via env vars [Daniel Belenky] stdci_runner: let mock_runner manage timeout [Greg Sheremeta] remove 4.1 change queue from ovirt-engine-nodejs config ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.verify_suspend_resume_vm0_42 Error Message: 0 != 1 -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:5c4788eb-8a69-4127-878e-8d49d9fb07c7:Get ssh client for lago-compat-4-2-suite-master-host-0: lago.ssh: DEBUG: end task:5c4788eb-8a69-4127-878e-8d49d9fb07c7:Get ssh client for lago-compat-4-2-suite-master-host-0: lago.ssh: DEBUG: Running 012e8ff2 on lago-compat-4-2-suite-master-host-0: awk 'BEGIN { n = 0; } $1 + " " + $2 > "IOError: [Errno" && $0 ~ /START create\(.*memoryDumpVolume/ { n = n + 1; } END { print n; }' /var/log/vdsm/vdsm.log lago.ssh: DEBUG: Command 012e8ff2 on lago-compat-4-2-suite-master-host-0 returned with 0 lago.ssh: DEBUG: Command 012e8ff2 on lago-compat-4-2-suite-master-host-0 output: 0 --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/004_basic_sanity.py", line 1140, in verify_suspend_resume_vm0 nt.assert_equals(log_line_count('START create\(.*' + identifier), 1) File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual raise self.failureException(msg) '0 != 1\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:5c4788eb-8a69-4127-878e-8d49d9fb07c7:Get ssh client for lago-compat-4-2-suite-master-host-0:\nlago.ssh: DEBUG: end task:5c4788eb-8a69-4127-878e-8d49d9fb07c7:Get ssh client for lago-compat-4-2-suite-master-host-0:\nlago.ssh: DEBUG: Running 012e8ff2 on lago-compat-4-2-suite-master-host-0: awk \'BEGIN { n = 0; } $1 + " " + $2 > "IOError: [Errno" && $0 ~ /START create\\(.*memoryDumpVolume/ { n = n + 1; } END { print n; }\' /var/log/vdsm/vdsm.log\nlago.ssh: DEBUG: Command 012e8ff2 on lago-compat-4-2-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command 012e8ff2 on lago-compat-4-2-suite-master-host-0 output:\n 0\n\n--------------------- >> end captured logging << ---------------------'

1 1

[JIRA] (OVIRT-2480) ovirt-srv04 out of disk space
by Evgheni Dereveanchin (oVirt JIRA) 06 Sep '18

06 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2480?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-2480: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > ovirt-srv04 out of disk space > ----------------------------- > > Key: OVIRT-2480 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2480 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > > ovirt-srv04 ran out of disk space again. Noticed this while working on monitoring for this box. Need to confirm what is using disk space and fix the issue. There are 4 kubevirt_kubevirt_standard-check-pr jobs currently running in containers on this hosts so they are likely to fail. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[JIRA] (OVIRT-2480) ovirt-srv04 out of disk space
by Evgheni Dereveanchin (oVirt JIRA) 06 Sep '18

06 Sep '18

Evgheni Dereveanchin created OVIRT-2480: ------------------------------------------- Summary: ovirt-srv04 out of disk space Key: OVIRT-2480 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2480 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Evgheni Dereveanchin Assignee: infra ovirt-srv04 ran out of disk space again. Noticed this while working on monitoring for this box. Need to confirm what is using disk space and fix the issue. There are 4 kubevirt_kubevirt_standard-check-pr jobs currently running in containers on this hosts so they are likely to fail. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[CQ]: 94160, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Sep '18

06 Sep '18

A system test invoked by the "ovirt-master" change queue including change 94160,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/94160/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10159/

1 0

[JIRA] (OVIRT-2476) move jira redirect vhost to OSAS
by Marc Dequènes (Duck) (oVirt JIRA) 06 Sep '18

06 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2476?page=com.atlassian.jira.… ] Marc Dequènes (Duck) reassigned OVIRT-2476: ------------------------------------------- Assignee: Marc Dequènes (Duck) (was: infra) > move jira redirect vhost to OSAS > -------------------------------- > > Key: OVIRT-2476 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2476 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Evgheni Dereveanchin > Assignee: Marc Dequènes (Duck) > > The jira.ovirt.org -> ovirt-jira.atlassian.net redirect is handled by a virtualhost located on monitoring.ovirt.org for historical reasons. Now that OSAS has a dedicated web host we can easily move it there to avoid mixing server roles. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100091)

1 0

[CQ]: 93777, 4 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Sep '18

06 Sep '18

A system test invoked by the "ovirt-master" change queue including change 93777,4 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/93777/4 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10156/

1 0

[CQ]: 94171, 3 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Sep '18

06 Sep '18

A system test invoked by the "ovirt-master" change queue including change 94171,3 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/94171/3 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10154/

1 0

[CQ]: 93842, 8 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Sep '18

06 Sep '18

A system test invoked by the "ovirt-master" change queue including change 93842,8 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/93842/8 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10152/

1 0

[CQ]: 93748, 9 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Sep '18

06 Sep '18

A system test invoked by the "ovirt-master" change queue including change 93748,9 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/93748/9 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10147/

1 0

oVirt infra daily report - unstable production jobs - 574
by jenkins＠jenkins.phx.ovirt.org 06 Sep '18

06 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/574//artifact/exported-a… Cheers, Jenkins

1 0

[CQ]: 94157, 3 (ovirt-imageio) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Sep '18

06 Sep '18

A system test invoked by the "ovirt-master" change queue including change 94157,3 (ovirt-imageio) failed. However, this change seems not to be the root cause for this failure. Change 93950,18 (ovirt-imageio) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 93950,18 (ovirt-imageio) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/94157/3 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/93950/18 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10142/

1 0

Change in stdci-staging[master]: Dummy patch to test timeouts
by review＠gerrit-staging.phx.ovirt.org 05 Sep '18

05 Sep '18

From Barak Korren <bkorren(a)redhat.com>: Hello Jenkins CI, I'd like you to reexamine a change. Please visit https://gerrit-staging.phx.ovirt.org/212 to look at the new patch set (#3). Change subject: Dummy patch to test timeouts ...................................................................... Dummy patch to test timeouts Makes check-patch just sleep for 30 seconds Change-Id: Iacafe73089d8f51a05ee590a284107bd3f842f72 Signed-off-by: Barak Korren <bkorren(a)redhat.com> --- M automation/check-patch.sh D automation/check-patch.yaml D stdci.yaml 3 files changed, 4 insertions(+), 19 deletions(-) git pull ssh://gerrit-staging.phx.ovirt.org:29418/stdci-staging refs/changes/12/212/3 -- To view, visit https://gerrit-staging.phx.ovirt.org/212 To unsubscribe, visit https://gerrit-staging.phx.ovirt.org/settings Gerrit-Project: stdci-staging Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iacafe73089d8f51a05ee590a284107bd3f842f72 Gerrit-Change-Number: 212 Gerrit-PatchSet: 3 Gerrit-Owner: Barak Korren <bkorren(a)redhat.com> Gerrit-Reviewer: Barak Korren <bkorren(a)redhat.com> Gerrit-Reviewer: Jenkins CI <infra(a)ovirt.org>

1 0

Change in stdci-staging[master]: Dummy patch to test timeouts
by review＠gerrit-staging.phx.ovirt.org 05 Sep '18

05 Sep '18

From Barak Korren <bkorren(a)redhat.com>: Hello Jenkins CI, I'd like you to reexamine a change. Please visit https://gerrit-staging.phx.ovirt.org/212 to look at the new patch set (#2). Change subject: Dummy patch to test timeouts ...................................................................... Dummy patch to test timeouts Makes check-patch just sleep for 30 seconds Change-Id: Iacafe73089d8f51a05ee590a284107bd3f842f72 Signed-off-by: Barak Korren <bkorren(a)redhat.com> --- M automation/check-patch.sh D stdci.yaml 2 files changed, 4 insertions(+), 8 deletions(-) git pull ssh://gerrit-staging.phx.ovirt.org:29418/stdci-staging refs/changes/12/212/2 -- To view, visit https://gerrit-staging.phx.ovirt.org/212 To unsubscribe, visit https://gerrit-staging.phx.ovirt.org/settings Gerrit-Project: stdci-staging Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iacafe73089d8f51a05ee590a284107bd3f842f72 Gerrit-Change-Number: 212 Gerrit-PatchSet: 2 Gerrit-Owner: Barak Korren <bkorren(a)redhat.com> Gerrit-Reviewer: Barak Korren <bkorren(a)redhat.com> Gerrit-Reviewer: Jenkins CI <infra(a)ovirt.org>

1 0

[CQ]: 94106, 5 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 05 Sep '18

05 Sep '18

A system test invoked by the "ovirt-master" change queue including change 94106,5 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 94081,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94081,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/94106/5 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94081/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10139/

1 0

[oVirt CI] standard-enqueue - Build #16289 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 05 Sep '18

05 Sep '18

Build: http://jenkins.ovirt.org/job/standard-enqueue/16289/ Build Name: #16289 Build Description: Gerrit: 94165 - ovirt-engine-nodejs-modules (4.1, 4.2, master) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/94165 - title: add pre-seed for ovirt-web-ui - project: ovirt-engine-nodejs-modules - branch: master - author: Greg Sheremeta <gshereme(a)redhat.com>

1 0