- Infra - oVirt List Archives

[CQ]: 114035, 4 (ovirt-system-tests) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 30 Mar '21

30 Mar '21

A system test invoked by the "ovirt-master" change queue including change 114035,4 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 113144,5 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 113144,5 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/114035/4 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/113144/5 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/28850/

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 338 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Mar '21

30 Mar '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/33… Build Number: 338 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #338 [Martin Perina] network: Readd network-suite-4.3 from check-patch ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.run_vms Error Message: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 694, in run_vms vm0_service.start(use_cloud_init=True, vm=vm_params) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 32520, in start return self._internal_action(action, 'start', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 134, in _check_fault self._raise_error(response, body.fault) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_el7-nightly - Build # 331 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Mar '21

30 Mar '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build Number: 331 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #331 [Martin Perina] network: Readd network-suite-4.3 from check-patch ----------------- Failed Tests: ----------------- No tests ran.

1 2

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1002 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Mar '21

30 Mar '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1002 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1002 [Dominik Holler] network: Remove line break before binary operator ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_005_network_by_label.test_assign_hosts_network_label Error Message: ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot add Label. Operation can be performed only when Host status is Maintenance, Up, NonOperational.]". HTTP response code is 409. Stack Trace: system_service = <ovirtsdk4.services.SystemService object at 0x7fbab1fecfd0> hosts_service = <ovirtsdk4.services.HostsService object at 0x7fbab1fec6a0> ost_cluster_name = 'test-cluster' def test_assign_hosts_network_label( system_service, hosts_service, ost_cluster_name): """ Assigns NETWORK_LABEL to first network interface of every host in cluster """ def _assign_host_network_label(host): host_service = hosts_service.host_service(id=host.id) nics_service = host_service.nics_service() nics = sorted(nics_service.list(), key=lambda n: n.name) assert len(nics) >= 1 nic = nics[0] nic_service = nics_service.nic_service(id=nic.id) labels_service = nic_service.network_labels_service() return labels_service.add(NetworkLabel(id=NETWORK_LABEL, host_nic=nic)) hosts = test_utils.hosts_in_cluster_v4(system_service, ost_cluster_name) vec = utils.func_vector(_assign_host_network_label, [(h,) for h in hosts]) vt = utils.VectorThread(vec) vt.start_all() > assert all(vt.join_all()) basic-suite-master/test-scenarios/test_005_network_by_label.py:67: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ost_utils/ost_utils/utils.py:73: in join_all raise exc_info[1].with_traceback(exc_info[2]) ost_utils/ost_utils/utils.py:34: in _ret_via_queue queue.put({'return': func()}) basic-suite-master/test-scenarios/test_005_network_by_label.py:61: in _assign_host_network_label return labels_service.add(NetworkLabel(id=NETWORK_LABEL, host_nic=nic)) /usr/lib64/python3.6/site-packages/ovirtsdk4/services.py:17454: in add return self._internal_add(label, headers, query, wait) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:232: in _internal_add return future.wait() if wait else future /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:229: in callback self._check_fault(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:132: in _check_fault self._raise_error(response, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7fbab292a668> detail = <ovirtsdk4.types.Fault object at 0x7fbab292a6a0> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot add Label. Operation can be performed only when Host status is Maintenance, Up, NonOperational.]". HTTP response code is 409. /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:118: Error

1 1

oVirt infra daily report - unstable production jobs - 1510
by jenkins＠jenkins.phx.ovirt.org 30 Mar '21

30 Mar '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1510//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 336 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Mar '21

29 Mar '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/33… Build Number: 336 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #336 [Martin Perina] network: Readd network-suite-4.3 from check-patch ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.run_vms Error Message: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 694, in run_vms vm0_service.start(use_cloud_init=True, vm=vm_params) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 32520, in start return self._internal_action(action, 'start', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 134, in _check_fault self._raise_error(response, body.fault) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_el7-nightly - Build # 328 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Mar '21

29 Mar '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build Number: 328 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #328 [Martin Perina] network: Readd network-suite-4.3 from check-patch ----------------- Failed Tests: ----------------- No tests ran.

1 2

[JIRA] (OVIRT-3091) [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://mirror.slu.cz/ovirt
by Alertmanager_Bot (oVirt JIRA) 29 Mar '21

29 Mar '21

Alertmanager_Bot created OVIRT-3091: --------------------------------------- Summary: [FIRING:1] MirrorIsOutOfSync web-ovirt-mirrorchecker.apps.ovirt.org:80 oVirt-mirrors-health http://mirror.slu.cz/ovirt Key: OVIRT-3091 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3091 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Alertmanager_Bot Assignee: infra Labels: - alertname = MirrorIsOutOfSync - instance = web-ovirt-mirrorchecker.apps.ovirt.org:80 - job = oVirt-mirrors-health - url = http://mirror.slu.cz/ovirt Annotations: - description = http://mirror.slu.cz/ovirt (measured by oVirt-mirrors-health) has not been synced in the past two days. Source: http://prometheus-0:9090/graph?g0.expr=mirror_health_check+%3E+259200&g0.ta… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100156)

1 0

oVirt infra daily report - unstable production jobs - 1509
by jenkins＠jenkins.phx.ovirt.org 29 Mar '21

29 Mar '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1509//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_el7-nightly - Build # 325 - Failure!
by jenkins＠jenkins.phx.ovirt.org 28 Mar '21

28 Mar '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build Number: 325 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #325 [Martin Perina] network: Readd network-suite-4.3 from check-patch ----------------- Failed Tests: ----------------- No tests ran.

1 2