November 2020 - Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 626 - Failure!
by jenkins＠jenkins.phx.ovirt.org 30 Nov '20

30 Nov '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 626 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #626 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7ffa8da0f938> engine_api = <ovirtsdk4.Connection object at 0x7ffa8ec87c50> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME) > vm0_service.start(use_cloud_init=True, vm=vm_params) ../basic-suite-master/test-scenarios/test_004_basic_sanity.py:830: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7ffa8eca0c50> detail = <ovirtsdk4.types.Fault object at 0x7ffa8eca0cd0> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error

1 1

[oVirt Jenkins] cleanup-gated-dummy-master - Build #345 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 Nov '20

30 Nov '20

Build: https://jenkins.ovirt.org/job/cleanup-gated-dummy-master/345/ Build Name: #345 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1390
by jenkins＠jenkins.phx.ovirt.org 30 Nov '20

30 Nov '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1390//artifact/exported… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-mock-copr-fc30-x86_64 #1680
by jenkins＠jenkins.phx.ovirt.org 29 Nov '20

29 Nov '20

See <https://jenkins.ovirt.org/job/system-sync_mirrors-mock-copr-fc30-x86_64/168…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <https://jenkins.ovirt.org/job/system-sync_mirrors-mock-copr-fc30-x86_64/ws/> The recommended git tool is: NONE No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git --version # 'git version 1.8.3.1' > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 58b093b8f190e3eb8a759664cf1339da9cce04d1 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 58b093b8f190e3eb8a759664cf1339da9cce04d1 # timeout=10 Commit message: "remove upgrades suites and 4.3 from nightly." > git rev-list --no-walk 58b093b8f190e3eb8a759664cf1339da9cce04d1 # timeout=10 [Checks API] No suitable checks publisher found. [system-sync_mirrors-mock-copr-fc30-x86_64] $ /bin/bash -xe /tmp/jenkins6801271955646133930.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror mock-copr-fc30 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + YUM_PATH=/yum/ + MIRRORS_BASE_PREFIX=/var/www/html/repos/yum/ + REPOQUERY_HTTP_BASE=http://mirrors.phx.ovirt.org/repos/yum/ + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + MAX_AGE=14 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror mock-copr-fc30 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror mock-copr-fc30 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=mock-copr-fc30 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs mock-copr-fc30 yum + local repo_name=mock-copr-fc30 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/mock-copr-fc30 /var/www/html/repos/yum/mock-copr-fc30/base + modify_reposync_conf mock-copr-fc30 jenkins/data/mirrors-reposync.conf + local repo_name=mock-copr-fc30 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local tmp_file=reposync.tmp + awk -v RS= -v 'ORS=\n\n' /main/ jenkins/data/mirrors-reposync.conf + awk -v RS= -v 'ORS=\n\n' /mock-copr-fc30/ jenkins/data/mirrors-reposync.conf + mv reposync.tmp jenkins/data/mirrors-reposync.conf + check_yum_sync_needed mock-copr-fc30 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=mock-copr-fc30 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/mock-copr-fc30 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync mock-copr-fc30 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=mock-copr-fc30 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=mock-copr-fc30 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/mock-copr-fc30/base --norepopath --urls --quiet Error setting up repositories: failure: repodata/repomd.xml from mock-copr-fc30: [Errno 256] No more mirrors to try. https://copr-be.cloud.fedoraproject.org/results/@mock/mock/fedora-30-x86_64…: [Errno 14] HTTPS Error 404 - Not Found + reposync_out= Build step 'Execute shell' marked build as failure

1 15

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 622 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Nov '20

29 Nov '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 622 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #622 [Michal Skrivanek] setup_playbook: disable gpg check for epel repo ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_import_vm1 Error Message: AssertionError: False != True after 600 seconds Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7f5d70158c50> @order_by(_TEST_LIST) def test_import_vm1(engine_api): engine = engine_api.system_service() sd = engine.storage_domains_service().list(search='name={}'.format(SD_ISCSI_NAME))[0] cluster = engine.clusters_service().list(search='name={}'.format(TEST_CLUSTER))[0] imports_service = engine.external_vm_imports_service() host = test_utils.get_first_active_host_by_name(engine) correlation_id = "test_validate_ova_import_vm" with engine_utils.wait_for_event(engine, 1165): # IMPORTEXPORT_STARTING_IMPORT_VM imports_service.add( types.ExternalVmImport( name=IMPORTED_VM_NAME, provider=types.ExternalVmProviderType.KVM, url=IMPORTED_OVA_NAME, cluster=types.Cluster( id=cluster.id ), storage_domain=types.StorageDomain( id=sd.id ), host=types.Host( id=host.id ), sparse=True > ), async=True, query={'correlation_id': correlation_id} ) ../basic-suite-master/test-scenarios/test_004_basic_sanity.py:712: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) ../ost_utils/ost_utils/engine_utils.py:44: in wait_for_event lambda: ../ost_utils/ost_utils/assertions.py:98: in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) ../ost_utils/ost_utils/assertions.py:83: in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ func = <function <lambda> at 0x7f5d7127ecf8>, value = True, timeout = 600 allowed_exceptions = [], initial_wait = 0 error_message = 'False != True after 600 seconds' def assert_equals_within( func, value, timeout, allowed_exceptions=None, initial_wait=10, error_message=None ): allowed_exceptions = allowed_exceptions or [] res = '<no-result-obtained>' with _EggTimer(timeout) as timer: while not timer.elapsed(): try: res = func() if res == value: return except Exception as exc: if _instance_of_any(exc, allowed_exceptions): time.sleep(3) continue LOGGER.exception("Unhandled exception in %s", func) raise if initial_wait == 0: time.sleep(3) else: time.sleep(initial_wait) initial_wait = 0 try: if error_message is None: error_message = '%s != %s after %s seconds' % (res, value, timeout) > raise AssertionError(error_message) E AssertionError: False != True after 600 seconds ../ost_utils/ost_utils/assertions.py:61: AssertionError

1 1

oVirt infra daily report - unstable production jobs - 1389
by jenkins＠jenkins.phx.ovirt.org 29 Nov '20

29 Nov '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1389//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 619 - Failure!
by jenkins＠jenkins.phx.ovirt.org 28 Nov '20

28 Nov '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 619 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #619 [Andrej Cernek] network: add remaining copy host network tests ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_run_vms Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. Stack Trace: assert_vm_is_alive = <function is_alive at 0x7f811d10b398> engine_api = <ovirtsdk4.Connection object at 0x7f811d108a90> management_gw_ip = '192.168.201.1' @order_by(_TEST_LIST) def test_run_vms(assert_vm_is_alive, engine_api, management_gw_ip): engine = engine_api.system_service() vm_params = types.Vm( initialization=types.Initialization( user_name=VM_USER_NAME, root_password=VM_PASSWORD ) ) vm_params.initialization.host_name = BACKUP_VM_NAME backup_vm_service = test_utils.get_vm_service(engine, BACKUP_VM_NAME) backup_vm_service.start(use_cloud_init=True, vm=vm_params) vm_params.initialization.host_name = VM2_NAME vm2_service = test_utils.get_vm_service(engine, VM2_NAME) vm2_service.start(use_cloud_init=True, vm=vm_params) # CirrOS cloud-init is different, networking doesn't work since it doesn't support the format oVirt is using vm_params.initialization.host_name = VM0_NAME # hostname seems to work, the others not vm_params.initialization.dns_search = 'lago.local' vm_params.initialization.domain = 'lago.local' vm_params.initialization.dns_servers = management_gw_ip vm0_service = test_utils.get_vm_service(engine, VM0_NAME) > vm0_service.start(use_cloud_init=True, vm=vm_params) ../basic-suite-master/test-scenarios/test_004_basic_sanity.py:830: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:32520: in start return self._internal_action(action, 'start', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7f811e261e90> detail = <ovirtsdk4.types.Fault object at 0x7f811e261f10> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error

1 2

oVirt infra daily report - unstable production jobs - 1388
by jenkins＠jenkins.phx.ovirt.org 28 Nov '20

28 Nov '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1388//artifact/exported… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 616 - Failure!
by jenkins＠jenkins.phx.ovirt.org 27 Nov '20

27 Nov '20

Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Build Number: 616 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #616 [Eitan Raviv] network: remove lago-env arg ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: basic-suite-master.test-scenarios.test_007_sd_reattach.test_deactivate_storage_domain Error Message: Error: Fault reason is "Operation Failed". Fault detail is "[Cannot deactivate Storage while there are running tasks on this Storage. -Tasks running: UploadStream -Please wait until tasks will finish and try again.]". HTTP response code is 409. Stack Trace: engine_api = <ovirtsdk4.Connection object at 0x7fbc94081f90> @order_by(_TEST_LIST) def test_deactivate_storage_domain(engine_api): # TODO: uncomment once VnicSetup checks are fixed. # TODO: this also seems to leave running tasks behind which break the deactivation. # TODO: it should be tested in multiple runs or properly waited for. # VnicSetup.vnic_setup().init(engine_api.system_service(), # VM2_NAME, DC_NAME, CLUSTER_NAME) dc = test_utils.data_center_service(engine_api.system_service(), DC_NAME) > _get_storage_domain(dc, SD_SECOND_NFS_NAME, service=True).deactivate() ../basic-suite-master/test-scenarios/test_007_sd_reattach.py:85: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib64/python2.7/site-packages/ovirtsdk4/services.py:3392: in deactivate return self._internal_action(action, 'deactivate', None, headers, query, wait) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:299: in _internal_action return future.wait() if wait else future /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:55: in wait return self._code(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:296: in callback self._check_fault(response) /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:134: in _check_fault self._raise_error(response, body.fault) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ response = <ovirtsdk4.http.Response object at 0x7fbc94081b50> detail = <ovirtsdk4.types.Fault object at 0x7fbc94081a50> @staticmethod def _raise_error(response, detail=None): """ Creates and raises an error containing the details of the given HTTP response and fault. This method is intended for internal use by other components of the SDK. Refrain from using it directly, as backwards compatibility isn't guaranteed. """ fault = detail if isinstance(detail, types.Fault) else None msg = '' if fault: if fault.reason: if msg: msg += ' ' msg = msg + 'Fault reason is "%s".' % fault.reason if fault.detail: if msg: msg += ' ' msg = msg + 'Fault detail is "%s".' % fault.detail if response: if response.code: if msg: msg += ' ' msg = msg + 'HTTP response code is %s.' % response.code if response.message: if msg: msg += ' ' msg = msg + 'HTTP response message is "%s".' % response.message if isinstance(detail, six.string_types): if msg: msg += ' ' msg = msg + detail + '.' class_ = Error if response is not None: if response.code in [401, 403]: class_ = AuthError elif response.code == 404: class_ = NotFoundError error = class_(msg) error.code = response.code if response else None error.fault = fault > raise error E Error: Fault reason is "Operation Failed". Fault detail is "[Cannot deactivate Storage while there are running tasks on this Storage. E -Tasks running: UploadStream E -Please wait until tasks will finish and try again.]". HTTP response code is 409. /usr/lib64/python2.7/site-packages/ovirtsdk4/service.py:118: Error

1 2

oVirt infra daily report - unstable production jobs - 1387
by jenkins＠jenkins.phx.ovirt.org 27 Nov '20

27 Nov '20

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - https://jenkins.ovirt.org/job/system_jenkins-report/1387//artifact/exported… Cheers, Jenkins

1 0