July 2021 - Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 658 - Failure!
by jenkins＠jenkins.phx.ovirt.org 15 Jul '21

15 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/65… Build Number: 658 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #658 [parthdhanjal] Updating max click version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.run_vms Error Message: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 694, in run_vms vm0_service.start(use_cloud_init=True, vm=vm_params) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 32520, in start return self._internal_action(action, 'start', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 134, in _check_fault self._raise_error(response, body.fault) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.

1 1

[oVirt Jenkins] deploy-to-gated-dummy-master - Build #569 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 15 Jul '21

15 Jul '21

Build: https://jenkins.ovirt.org/job/deploy-to-gated-dummy-master/569/ Build Name: #569 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 1

oVirt infra daily report - unstable production jobs - 1617
by jenkins＠jenkins.phx.ovirt.org 15 Jul '21

15 Jul '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1617//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1317 - Failure!
by jenkins＠jenkins.phx.ovirt.org 14 Jul '21

14 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1317 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1317 [Saif Abu Saleh] docs: Improve First Run documentation [Sandro Bonazzola] ovirt-log-collector: drop 4.3 jobs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_cold_incremental_backup_vm2 Error Message: ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM: Disk is locked. Please try again later.]". HTTP response code is 409. Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f2f3f87c5f8> get_vm_service_for_vm = <function get_vm_service_for_vm.<locals>.service_for at 0x7f2f3d5441e0> @order_by(_TEST_LIST) def test_cold_incremental_backup_vm2(engine_api, get_vm_service_for_vm): _verify_vm_state(engine_api.system_service(), VM2_NAME, types.VmStatus.DOWN) vm2_backups_service = get_vm_service_for_vm(VM2_NAME).backups_service() backup.perform_incremental_vm_backup( > engine_api, vm2_backups_service, DISK2_NAME, "cold_vm_backup") basic-suite-master/test-scenarios/test_004_basic_sanity.py:1045: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ost_utils/ost_utils/storage_utils/backup.py:75: in perform_incremental_vm_backup correlation_id="incremental_" + correlation_id) ost_utils/ost_utils/storage_utils/backup.py:34: in perform_vm_backup ), query={'correlation_id': correlation_id} /usr/lib64/python3.6/site-packages/ovirtsdk4/services.py:34139: in add return self._internal_add(backup, headers, query, wait) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:232: in _internal_add return future.wait() if wait else future /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:229: in callback self._check_fault(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:132: in _check_fault self._raise_error(response, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f2f3d5e22b0> detail = <ovirtsdk4.types.Fault object at 0x7f2f3d5e2b70> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM: Disk is locked. Please try again later.]". HTTP response code is 409. /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:118: Error

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 655 - Failure!
by jenkins＠jenkins.phx.ovirt.org 14 Jul '21

14 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/65… Build Number: 655 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #655 [parthdhanjal] Updating max click version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.run_vms Error Message: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 694, in run_vms vm0_service.start(use_cloud_init=True, vm=vm_params) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 32520, in start return self._internal_action(action, 'start', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 134, in _check_fault self._raise_error(response, body.fault) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.

1 1

oVirt infra daily report - unstable production jobs - 1616
by jenkins＠jenkins.phx.ovirt.org 14 Jul '21

14 Jul '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1616//artifact/exported… Cheers, Jenkins

1 0

[CQ]: 115648, 2 (jenkins) failed "dummy-master" system tests, but isn't the failure root cause
by oVirt Jenkins 13 Jul '21

13 Jul '21

A system test invoked by the "dummy-master" change queue including change 115648,2 (jenkins) failed. However, this change seems not to be the root cause for this failure. Change 115395,2 (jenkins) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 115395,2 (jenkins) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/c/jenkins/+/#/c/115648/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/c/jenkins/+/#/c/115395/2 For failed test results see: https://jenkins.ovirt.org/job/dummy-master_change-queue-tester/645/

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1314 - Failure!
by jenkins＠jenkins.phx.ovirt.org 13 Jul '21

13 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1314 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1314 [Michal Skrivanek] remove obsolete pkgs.txt files ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_cold_incremental_backup_vm2 Error Message: ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM: Disk is locked. Please try again later.]". HTTP response code is 409. Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f8893481198> get_vm_service_for_vm = <function get_vm_service_for_vm.<locals>.service_for at 0x7f889109e0d0> @order_by(_TEST_LIST) def test_cold_incremental_backup_vm2(engine_api, get_vm_service_for_vm): _verify_vm_state(engine_api.system_service(), VM2_NAME, types.VmStatus.DOWN) vm2_backups_service = get_vm_service_for_vm(VM2_NAME).backups_service() backup.perform_incremental_vm_backup( > engine_api, vm2_backups_service, DISK2_NAME, "cold_vm_backup") basic-suite-master/test-scenarios/test_004_basic_sanity.py:1051: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ost_utils/ost_utils/storage_utils/backup.py:75: in perform_incremental_vm_backup correlation_id="incremental_" + correlation_id) ost_utils/ost_utils/storage_utils/backup.py:34: in perform_vm_backup ), query={'correlation_id': correlation_id} /usr/lib64/python3.6/site-packages/ovirtsdk4/services.py:34139: in add return self._internal_add(backup, headers, query, wait) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:232: in _internal_add return future.wait() if wait else future /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:229: in callback self._check_fault(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:132: in _check_fault self._raise_error(response, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f88910d4400> detail = <ovirtsdk4.types.Fault object at 0x7f88910d4390> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM: Disk is locked. Please try again later.]". HTTP response code is 409. /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:118: Error

1 1

oVirt infra daily report - unstable production jobs - 1615
by jenkins＠jenkins.phx.ovirt.org 13 Jul '21

13 Jul '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1615//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3110) OST failure, bad comment
by Michal Skrivanek (oVirt JIRA) 12 Jul '21

12 Jul '21

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3110?page=com.atlassian.jira.… ] Michal Skrivanek commented on OVIRT-3110: ----------------------------------------- > On 10. 7. 2021, at 0:14, Nir Soffer <nsoffer(a)redhat.com> wrote: > > On Sat, Jul 10, 2021 at 12:59 AM Nir Soffer <nsoffer(a)redhat.com> wrote: >> >> OST failed, and posted unprocessed comment: >> >> OST Verified -1 >> Patch Set 1: Verified-1 >> https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 : >> OST https://redir.apps.ovirt.org/dj/job/ds-ost-baremetal_manual/$OST_NUMBER >> failed with: $OST_MESSAGE >> >> https://gerrit.ovirt.org/c/vdsm/+/115642#message-b2993761_adbc1b6a >> >> Looking at: >> https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 >> >> It looks like environment issue: >> >> ERROR: Build step failed with exception >> >> java.lang.NullPointerException: no workspace from node >> hudson.slaves.DumbSlave[hpe-bl280g6-12...] which is computer >> hudson.slaves.SlaveComputer@2e2cb321 and has channel null >> >> So we have 2 issues: >> 1. If build fails, error is not handled properly producing bad comment >> 2. the actual build failure > > I started another build manually (8543) and it started normally, so this was > probably a temporary issue. Seems someone rebooted the jenkins host in the middle of the run > >> >> The first issue should be handled by posting a better comment: >> >> OST build error: >> https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 >> >> And OST should not mark the patch as verified -1, since it did not run any test. It’s a shell script that runs after the job that fails on missing workspace, it would be possible to set unstable status on a failure of that script, but It doesn’t seem to be possible to handle internal jenkins exception >> >> Marking a patch as -1 means some tests have failed, and the author >> needs to check >> why. A build error means OST or CI maintainers need to check why the >> error happened. > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/H4Q2MT7R7DXKW… > OST failure, bad comment > ------------------------ > > Key: OVIRT-3110 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3110 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > OST failed, and posted unprocessed comment: > OST Verified -1 > Patch Set 1: Verified-1 > https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 : > OST https://redir.apps.ovirt.org/dj/job/ds-ost-baremetal_manual/$OST_NUMBER > failed with: $OST_MESSAGE > https://gerrit.ovirt.org/c/vdsm/+/115642#message-b2993761_adbc1b6a > Looking at: > https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 > It looks like environment issue: > ERROR: Build step failed with exception > java.lang.NullPointerException: no workspace from node > hudson.slaves.DumbSlave[hpe-bl280g6-12...] which is computer > hudson.slaves.SlaveComputer@2e2cb321 and has channel null > So we have 2 issues: > 1. If build fails, error is not handled properly producing bad comment > 2. the actual build failure > The first issue should be handled by posting a better comment: > OST build error: > https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 > And OST should not mark the patch as verified -1, since it did not run any test. > Marking a patch as -1 means some tests have failed, and the author > needs to check > why. A build error means OST or CI maintainers need to check why the > error happened. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100168)

1 0