- Infra - oVirt List Archives

oVirt infra daily report - unstable production jobs - 1615
by jenkins＠jenkins.phx.ovirt.org 13 Jul '21

13 Jul '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1615//artifact/exported… Cheers, Jenkins

1 0

[JIRA] (OVIRT-3110) OST failure, bad comment
by Michal Skrivanek (oVirt JIRA) 12 Jul '21

12 Jul '21

[ https://ovirt-jira.atlassian.net/browse/OVIRT-3110?page=com.atlassian.jira.… ] Michal Skrivanek commented on OVIRT-3110: ----------------------------------------- > On 10. 7. 2021, at 0:14, Nir Soffer <nsoffer(a)redhat.com> wrote: > > On Sat, Jul 10, 2021 at 12:59 AM Nir Soffer <nsoffer(a)redhat.com> wrote: >> >> OST failed, and posted unprocessed comment: >> >> OST Verified -1 >> Patch Set 1: Verified-1 >> https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 : >> OST https://redir.apps.ovirt.org/dj/job/ds-ost-baremetal_manual/$OST_NUMBER >> failed with: $OST_MESSAGE >> >> https://gerrit.ovirt.org/c/vdsm/+/115642#message-b2993761_adbc1b6a >> >> Looking at: >> https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 >> >> It looks like environment issue: >> >> ERROR: Build step failed with exception >> >> java.lang.NullPointerException: no workspace from node >> hudson.slaves.DumbSlave[hpe-bl280g6-12...] which is computer >> hudson.slaves.SlaveComputer@2e2cb321 and has channel null >> >> So we have 2 issues: >> 1. If build fails, error is not handled properly producing bad comment >> 2. the actual build failure > > I started another build manually (8543) and it started normally, so this was > probably a temporary issue. Seems someone rebooted the jenkins host in the middle of the run > >> >> The first issue should be handled by posting a better comment: >> >> OST build error: >> https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 >> >> And OST should not mark the patch as verified -1, since it did not run any test. It’s a shell script that runs after the job that fails on missing workspace, it would be possible to set unstable status on a failure of that script, but It doesn’t seem to be possible to handle internal jenkins exception >> >> Marking a patch as -1 means some tests have failed, and the author >> needs to check >> why. A build error means OST or CI maintainers need to check why the >> error happened. > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/H4Q2MT7R7DXKW… > OST failure, bad comment > ------------------------ > > Key: OVIRT-3110 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-3110 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > OST failed, and posted unprocessed comment: > OST Verified -1 > Patch Set 1: Verified-1 > https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 : > OST https://redir.apps.ovirt.org/dj/job/ds-ost-baremetal_manual/$OST_NUMBER > failed with: $OST_MESSAGE > https://gerrit.ovirt.org/c/vdsm/+/115642#message-b2993761_adbc1b6a > Looking at: > https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 > It looks like environment issue: > ERROR: Build step failed with exception > java.lang.NullPointerException: no workspace from node > hudson.slaves.DumbSlave[hpe-bl280g6-12...] which is computer > hudson.slaves.SlaveComputer@2e2cb321 and has channel null > So we have 2 issues: > 1. If build fails, error is not handled properly producing bad comment > 2. the actual build failure > The first issue should be handled by posting a better comment: > OST build error: > https://redir.apps.ovirt.org/dj/job/ds-ost-gating-trigger2/313 > And OST should not mark the patch as verified -1, since it did not run any test. > Marking a patch as -1 means some tests have failed, and the author > needs to check > why. A build error means OST or CI maintainers need to check why the > error happened. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100168)

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 650 - Failure!
by jenkins＠jenkins.phx.ovirt.org 12 Jul '21

12 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/65… Build Number: 650 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #650 [parthdhanjal] Updating max click version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.run_vms Error Message: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 694, in run_vms vm0_service.start(use_cloud_init=True, vm=vm_params) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 32520, in start return self._internal_action(action, 'start', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 134, in _check_fault self._raise_error(response, body.fault) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.

1 2

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1312 - Failure!
by jenkins＠jenkins.phx.ovirt.org 12 Jul '21

12 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1312 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1312 [hbraha] basic: uncomment vnic test in storage ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_cold_incremental_backup_vm2 Error Message: ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM: Disk is locked. Please try again later.]". HTTP response code is 409. Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f2b7098ca20> get_vm_service_for_vm = <function get_vm_service_for_vm.<locals>.service_for at 0x7f2b6e733ae8> @order_by(_TEST_LIST) def test_cold_incremental_backup_vm2(engine_api, get_vm_service_for_vm): _verify_vm_state(engine_api.system_service(), VM2_NAME, types.VmStatus.DOWN) vm2_backups_service = get_vm_service_for_vm(VM2_NAME).backups_service() backup.perform_incremental_vm_backup( > engine_api, vm2_backups_service, DISK2_NAME, "cold_vm_backup") basic-suite-master/test-scenarios/test_004_basic_sanity.py:1051: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ost_utils/ost_utils/storage_utils/backup.py:75: in perform_incremental_vm_backup correlation_id="incremental_" + correlation_id) ost_utils/ost_utils/storage_utils/backup.py:34: in perform_vm_backup ), query={'correlation_id': correlation_id} /usr/lib64/python3.6/site-packages/ovirtsdk4/services.py:34139: in add return self._internal_add(backup, headers, query, wait) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:232: in _internal_add return future.wait() if wait else future /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:229: in callback self._check_fault(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:132: in _check_fault self._raise_error(response, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f2b6e5b9320> detail = <ovirtsdk4.types.Fault object at 0x7f2b6e5b9be0> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM: Disk is locked. Please try again later.]". HTTP response code is 409. /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:118: Error

1 1

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 1310 - Failure!
by jenkins＠jenkins.phx.ovirt.org 12 Jul '21

12 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 1310 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #1310 [Yedidyah Bar David] he: de-duplicate and randomize MAC address ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_cold_incremental_backup_vm2 Error Message: ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM. The VM is during a backup operation.]". HTTP response code is 409. Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f397dc8f630> get_vm_service_for_vm = <function get_vm_service_for_vm.<locals>.service_for at 0x7f396f970f28> @order_by(_TEST_LIST) def test_cold_incremental_backup_vm2(engine_api, get_vm_service_for_vm): _verify_vm_state(engine_api.system_service(), VM2_NAME, types.VmStatus.DOWN) vm2_backups_service = get_vm_service_for_vm(VM2_NAME).backups_service() backup.perform_incremental_vm_backup( > engine_api, vm2_backups_service, DISK2_NAME, "cold_vm_backup") basic-suite-master/test-scenarios/test_004_basic_sanity.py:1051: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ost_utils/ost_utils/storage_utils/backup.py:75: in perform_incremental_vm_backup correlation_id="incremental_" + correlation_id) ost_utils/ost_utils/storage_utils/backup.py:34: in perform_vm_backup ), query={'correlation_id': correlation_id} /usr/lib64/python3.6/site-packages/ovirtsdk4/services.py:34139: in add return self._internal_add(backup, headers, query, wait) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:232: in _internal_add return future.wait() if wait else future /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:229: in callback self._check_fault(response) /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:132: in _check_fault self._raise_error(response, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f396f83c4e0> detail = <ovirtsdk4.types.Fault object at 0x7f396f83cb00> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot backup VM. The VM is during a backup operation.]". HTTP response code is 409. /usr/lib64/python3.6/site-packages/ovirtsdk4/service.py:118: Error

1 1

oVirt infra daily report - unstable production jobs - 1614
by jenkins＠jenkins.phx.ovirt.org 12 Jul '21

12 Jul '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1614//artifact/exported… Cheers, Jenkins

1 0

[CQ]: 113894, 18 (ovirt-system-tests) failed "ovirt-master" system tests
by oVirt Jenkins 11 Jul '21

11 Jul '21

Change 113894,18 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/c/ovirt-system-tests/+/#/c/113894/18 For failed test results see: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/30701/

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-4.3_nightly - Build # 647 - Failure!
by jenkins＠jenkins.phx.ovirt.org 11 Jul '21

11 Jul '21

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-4.3_nightly/64… Build Number: 647 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #647 [parthdhanjal] Updating max click version ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.run_vms Error Message: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_basic-suite-4.3_nightly/ovirt-system-tests/basic-suite-4.3/test-scenarios/004_basic_sanity.py", line 694, in run_vms vm0_service.start(use_cloud_init=True, vm=vm_params) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 32520, in start return self._internal_action(action, 'start', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 134, in _check_fault self._raise_error(response, body.fault) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.

1 1

oVirt infra daily report - unstable production jobs - 1613
by jenkins＠jenkins.phx.ovirt.org 11 Jul '21

11 Jul '21

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1613//artifact/exported… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el8-x86_64 #1396
by jenkins＠jenkins.phx.ovirt.org 10 Jul '21

10 Jul '21

See <https://jenkins.ovirt.org/job/system-sync_mirrors-epel-el8-x86_64/1396/disp…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <https://jenkins.ovirt.org/job/system-sync_mirrors-epel-el8-x86_64/ws/> The recommended git tool is: NONE No credentials specified > git rev-parse --resolve-git-dir <https://jenkins.ovirt.org/job/system-sync_mirrors-epel-el8-x86_64/ws/jenkin…> # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git --version # 'git version 1.8.3.1' > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/changes/49/110649/1:shlomi # timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision e4127ce8ca71892dec377f31aa677f9c4714dbc5 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e4127ce8ca71892dec377f31aa677f9c4714dbc5 # timeout=10 Commit message: "stdci_runner: added the rhel labels selection." > git rev-list --no-walk e4127ce8ca71892dec377f31aa677f9c4714dbc5 # timeout=10 [system-sync_mirrors-epel-el8-x86_64] $ /bin/bash -xe /tmp/jenkins6104159384595697709.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el8 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + YUM_PATH=/yum/ + MIRRORS_BASE_PREFIX=/var/www/html/repos/yum/ + REPOQUERY_HTTP_BASE=http://mirrors.phx.ovirt.org/repos/yum/ + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + MAX_AGE=14 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror epel-el8 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror epel-el8 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=epel-el8 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs epel-el8 yum + local repo_name=epel-el8 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/epel-el8 /var/www/html/repos/yum/epel-el8/base + modify_reposync_conf epel-el8 jenkins/data/mirrors-reposync.conf + local repo_name=epel-el8 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local tmp_file=reposync.tmp + awk -v RS= -v 'ORS=\n\n' /main/ jenkins/data/mirrors-reposync.conf + awk -v RS= -v 'ORS=\n\n' /epel-el8/ jenkins/data/mirrors-reposync.conf + mv reposync.tmp jenkins/data/mirrors-reposync.conf + check_yum_sync_needed epel-el8 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=epel-el8 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/epel-el8 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync epel-el8 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=epel-el8 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=epel-el8 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/epel-el8/base --norepopath --urls --quiet Error setting up repositories: failure: repodata/071b5f2aafc9104cfeda08c5dd51ab9dabf48cad38866ada4d7aee9306ab74f2-primary.sqlite.xz from epel-el8: [Errno 256] No more mirrors to try. http://mirror.math.princeton.edu/pub/epel/8/Everything/x86_64/repodata/071b…: [Errno 12] Timeout on http://mirror.math.princeton.edu/pub/epel/8/Everything/x86_64/repodata/071b…: (28, 'Connection timed out after 30001 milliseconds') http://mirror.arizona.edu/fedora-epel/8/Everything/x86_64/repodata/071b5f2a…: [Errno 14] HTTP Error 404 - Not Found http://mirrors.sonic.net/epel/8/Everything/x86_64/repodata/071b5f2aafc9104c…: [Errno 14] HTTP Error 404 - Not Found http://mirror.dal.nexril.net/fedora-epel/8/Everything/x86_64/repodata/071b5…: [Errno 14] HTTP Error 404 - Not Found https://mirror.atl.genesisadaptive.com/epel/8/Everything/x86_64/repodata/07…: [Errno 14] HTTPS Error 404 - Not Found http://mirror.steadfastnet.com/epel/8/Everything/x86_64/repodata/071b5f2aaf…: [Errno 14] HTTP Error 404 - Not Found http://mirrors.kernel.org/fedora-epel/8/Everything/x86_64/repodata/071b5f2a…: [Errno 14] HTTP Error 404 - Not Found http://mirror.sfo12.us.leaseweb.net/epel/8/Everything/x86_64/repodata/071b5…: [Errno 14] HTTP Error 404 - Not Found https://iad.mirror.rackspace.com/epel/8/Everything/x86_64/repodata/071b5f2a…: [Errno 14] HTTPS Error 404 - Not Found http://mirrors.syringanetworks.net/fedora-epel/8/Everything/x86_64/repodata…: [Errno 14] HTTP Error 404 - Not Found https://mirror.lax.genesisadaptive.com/epel/8/Everything/x86_64/repodata/07…: [Errno 14] HTTPS Error 404 - Not Found https://epel.mirror.shastacoe.net/epel/8/Everything/x86_64/repodata/071b5f2…: [Errno 14] HTTPS Error 404 - Not Found http://mirror.mrjester.net/fedora/epel/8/Everything/x86_64/repodata/071b5f2…: [Errno 14] HTTP Error 404 - Not Found http://ord.mirror.rackspace.com/epel/8/Everything/x86_64/repodata/071b5f2aa…: [Errno 14] HTTP Error 404 - Not Found https://mirror.genesisadaptive.com/epel/8/Everything/x86_64/repodata/071b5f…: [Errno 14] HTTPS Error 404 - Not Found https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/repoda…: [Errno 14] HTTPS Error 404 - Not Found http://mirrors.xmission.com/fedora-epel/8/Everything/x86_64/repodata/071b5f…: [Errno 14] HTTP Error 404 - Not Found https://d2lzkl7pfhq30w.cloudfront.net/pub/epel/8/Everything/x86_64/repodata…: [Errno 14] HTTPS Error 404 - Not Found http://mirrors.mit.edu/epel/8/Everything/x86_64/repodata/071b5f2aafc9104cfe…: [Errno 14] HTTP Error 404 - Not Found http://mirror.rnet.missouri.edu/epel/8/Everything/x86_64/repodata/071b5f2aa…: [Errno 14] HTTP Error 404 - Not Found http://mirror.team-cymru.com/epel/8/Everything/x86_64/repodata/071b5f2aafc9…: [Errno 14] HTTP Error 404 - Not Found http://mirror.cs.pitt.edu/epel/8/Everything/x86_64/repodata/071b5f2aafc9104…: [Errno 14] HTTP Error 404 - Not Found http://packages.oit.ncsu.edu/epel/8/Everything/x86_64/repodata/071b5f2aafc9…: [Errno 14] HTTP Error 404 - Not Found https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/re…: [Errno 14] HTTPS Error 404 - Not Found http://dfw.mirror.rackspace.com/epel/8/Everything/x86_64/repodata/071b5f2aa…: [Errno 14] HTTP Error 404 - Not Found https://pubmirror1.math.uh.edu/fedora-buffet/epel/8/Everything/x86_64/repod…: [Errno 14] HTTPS Error 404 - Not Found https://mirror.vcu.edu/pub/gnu+linux/epel/8/Everything/x86_64/repodata/071b…: [Errno 14] HTTPS Error 404 - Not Found http://mirrors.liquidweb.com/fedora-epel/8/Everything/x86_64/repodata/071b5…: [Errno 14] HTTP Error 404 - Not Found https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/repodata/071b5f2a…: [Errno 14] HTTPS Error 404 - Not Found http://mirror.its.dal.ca/pub/epel/8/Everything/x86_64/repodata/071b5f2aafc9…: [Errno 14] HTTP Error 404 - Not Found https://mirror.csclub.uwaterloo.ca/fedora/epel/8/Everything/x86_64/repodata…: [Errno 14] HTTPS Error 404 - Not Found http://fedora-epel.mirror.iweb.com/8/Everything/x86_64/repodata/071b5f2aafc…: [Errno 14] HTTP Error 404 - Not Found http://mirrors.upr.edu/epel/8/Everything/x86_64/repodata/071b5f2aafc9104cfe…: [Errno 14] HTTP Error 404 - Not Found http://mirror.dst.ca/epel/8/Everything/x86_64/repodata/071b5f2aafc9104cfeda…: [Errno 14] HTTP Error 404 - Not Found + reposync_out= Build step 'Execute shell' marked build as failure

1 1