- Infra - oVirt List Archives

oVirt infra daily report - unstable production jobs - 1584
by jenkins＠jenkins.phx.ovirt.org 12 Jun '21

12 Jun '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1584//artifact/exported… Cheers, Jenkins

1 0

[CQ]: 115204, 1 (ovirt-system-tests) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 11 Jun '21

11 Jun '21

A system test invoked by the "ovirt-master" change queue including change 115204,1 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 113144,5 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 113144,5 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/115204/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/113144/5 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/30160/

1 0

[CQ]: 115106, 3 (ovirt-system-tests) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 11 Jun '21

11 Jun '21

A system test invoked by the "ovirt-master" change queue including change 115106,3 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 113144,5 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 113144,5 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/115106/3 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/113144/5 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/30158/

1 0

[JIRA] (OVIRT-3097) CI failures due to oVirt proxy getting forbidden access to CentOS mirrors
by Sandro Bonazzola (oVirt JIRA) 11 Jun '21

11 Jun '21

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3097?page=com.atlassian.jira.… ] Sandro Bonazzola updated OVIRT-3097: ------------------------------------ Components: oVirt CI > CI failures due to oVirt proxy getting forbidden access to CentOS mirrors > ------------------------------------------------------------------------- > > Key: OVIRT-3097 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3097 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: oVirt CI > Reporter: Sandro Bonazzola > Assignee: infra > Priority: Highest > > Hi, > we are experiencing build failures due to > *08:49:10* # /usr/bin/yum --installroot > /var/lib/mock/epel-8-x86_64-6c17b26f3795335ce1ac45b0dcdbc9d5-bootstrap-8304/root/ > --releasever 8 install dnf dnf-plugins-core distribution-gpg-keys > --setopt=tsflags=nocontexts > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > Failed to set locale, defaulting to C > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml: > [Errno 14] HTTP Error 403 - Forbidden > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > Trying other mirror. > ... > *08:49:10* failure: repodata/repomd.xml from centos-appstream-el8: > [Errno 256] No more mirrors to try. > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml: > [Errno 14] HTTP Error 403 - Forbidden > (see https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…) > I checked our squid logs on proxy01.phx.ovirt.org and I see: > 1623395369.584 1 38.145.50.116 TCP_DENIED/403 4148 GET > http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml > - HIER_NONE/- text/html > Apparently there's no option in oVirt CI system to turn off the proxy > since mock_runner is always called with --try-proxy option. > Anyone that can look into this? > -- > Sandro Bonazzola > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://www.redhat.com/> > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours.* -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100166)

1 0

[JIRA] (OVIRT-3097) CI failures due to oVirt proxy getting forbidden access to CentOS mirrors
by Sandro Bonazzola (oVirt JIRA) 11 Jun '21

11 Jun '21

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3097?page=com.atlassian.jira.… ] Sandro Bonazzola updated OVIRT-3097: ------------------------------------ Priority: Highest (was: Medium) > CI failures due to oVirt proxy getting forbidden access to CentOS mirrors > ------------------------------------------------------------------------- > > Key: OVIRT-3097 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3097 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Sandro Bonazzola > Assignee: infra > Priority: Highest > > Hi, > we are experiencing build failures due to > *08:49:10* # /usr/bin/yum --installroot > /var/lib/mock/epel-8-x86_64-6c17b26f3795335ce1ac45b0dcdbc9d5-bootstrap-8304/root/ > --releasever 8 install dnf dnf-plugins-core distribution-gpg-keys > --setopt=tsflags=nocontexts > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > Failed to set locale, defaulting to C > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml: > [Errno 14] HTTP Error 403 - Forbidden > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > Trying other mirror. > ... > *08:49:10* failure: repodata/repomd.xml from centos-appstream-el8: > [Errno 256] No more mirrors to try. > <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* > http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml: > [Errno 14] HTTP Error 403 - Forbidden > (see https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…) > I checked our squid logs on proxy01.phx.ovirt.org and I see: > 1623395369.584 1 38.145.50.116 TCP_DENIED/403 4148 GET > http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml > - HIER_NONE/- text/html > Apparently there's no option in oVirt CI system to turn off the proxy > since mock_runner is always called with --try-proxy option. > Anyone that can look into this? > -- > Sandro Bonazzola > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://www.redhat.com/> > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours.* -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100166)

1 0

[JIRA] (OVIRT-3097) CI failures due to oVirt proxy getting forbidden access to CentOS mirrors
by Sandro Bonazzola (oVirt JIRA) 11 Jun '21

11 Jun '21

Sandro Bonazzola created OVIRT-3097: --------------------------------------- Summary: CI failures due to oVirt proxy getting forbidden access to CentOS mirrors Key: OVIRT-3097 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3097 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Sandro Bonazzola Assignee: infra Hi, we are experiencing build failures due to *08:49:10* # /usr/bin/yum --installroot /var/lib/mock/epel-8-x86_64-6c17b26f3795335ce1ac45b0dcdbc9d5-bootstrap-8304/root/ --releasever 8 install dnf dnf-plugins-core distribution-gpg-keys --setopt=tsflags=nocontexts <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* Failed to set locale, defaulting to C <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* Trying other mirror. ... *08:49:10* failure: repodata/repomd.xml from centos-appstream-el8: [Errno 256] No more mirrors to try. <https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…>*08:49:10* http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden (see https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el8-x8…) I checked our squid logs on proxy01.phx.ovirt.org and I see: 1623395369.584 1 38.145.50.116 TCP_DENIED/403 4148 GET http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/repomd.xml - HIER_NONE/- text/html Apparently there's no option in oVirt CI system to turn off the proxy since mock_runner is always called with --try-proxy option. Anyone that can look into this? -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.* -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100166)

1 0

[JIRA] (OVIRT-3096) Foreman SSL certificate expired
by Sandro Bonazzola (oVirt JIRA) 11 Jun '21

11 Jun '21

Sandro Bonazzola created OVIRT-3096: --------------------------------------- Summary: Foreman SSL certificate expired Key: OVIRT-3096 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3096 Project: oVirt - virtualization made easy Issue Type: Bug Components: oVirt Infra Reporter: Sandro Bonazzola Assignee: infra Priority: High Jenkins slaves are reporting: Jun 11 05:44:15 vm0035.phx.ovirt.org puppet-agent[31473]: Could not send report: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [certificate has expired for /CN=Puppet CA: foreman.ovirt.o -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100166)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1217 - Failure!
by jenkins＠jenkins.phx.ovirt.org 11 Jun '21

11 Jun '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1217 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1217 [Eitan Raviv] network: force select spm - wait for dc status ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_run_vms Error Message: AssertionError: False != True after 600 seconds Stack Trace: assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at 0x7fec55b807b8> engine_api = <ovirtsdk4.Connection object at 0x7fec55b90940> management_gw_ip = '192.168.200.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME) vm0_service.start(use_cloud_init=True, vm=vm_params) for vm_name in [VM0_NAME, BACKUP_VM_NAME]: > _verify_vm_state(engine, vm_name, types.VmStatus.UP) basic-suite-master/test-scenarios/test_004_basic_sanity.py:1089: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ basic-suite-master/test-scenarios/test_004_basic_sanity.py:212: in _verify_vm_state lambda: ost_utils/ost_utils/assertions.py:96: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ost_utils/ost_utils/assertions.py:81: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function _verify_vm_state.<locals>.<lambda> at 0x7fec55b80730> value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout) > raise AssertionError(error_message) E AssertionError: False != True after 600 seconds ost_utils/ost_utils/assertions.py:59: AssertionError

1 6

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_el7-nightly - Build # 550 - Failure!
by jenkins＠jenkins.phx.ovirt.org 11 Jun '21

11 Jun '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_el7-nightl… Build Number: 550 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #550 [parthdhanjal] Updating max click version ----------------- Failed Tests: ----------------- No tests ran.

1 1

oVirt infra daily report - unstable production jobs - 1583
by jenkins＠jenkins.phx.ovirt.org 11 Jun '21

11 Jun '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1583//artifact/exported… Cheers, Jenkins

1 0