December 2019 - Infra - oVirt List Archives

Container-based CI backend is now available for use
by Barak Korren 26 Mar '20

26 Mar '20

A little less then a month ago I sent an email to this list telling you all about ongoing work to create a new container-based backend for the oVirt CI system. I'm pleased to announce that we have managed to finally merged an initial set of patches implementing that backend yesterday, and it is now available for general use. *What? Were? How do I use it?* Documentation about how to use the new backend is now available in read the docs <https://ovirt-infra-docs.readthedocs.io/en/latest/CI/STDCI-Containers/index…> . *Wait! I needed it to do X which it doesn't!* For the time being the new backend lacks some features that some may consider to be essential, such as automated cloning of patch source code and build artifact collection. We already have implemented patches providing a substantial amount of additional functionality, and hopefully we will be able to merge them soon. Following is a list of those patches and the features they implement: 1. Automated source cloning support: - 104213 <https://gerrit.ovirt.org/104213>: Implement STDCI DSL support for initContainers - 104590 <https://gerrit.ovirt.org/104590>: STDCI DSL: Add the `decorate` option - 104668 <https://gerrit.ovirt.org/104668>: Document source cloning extension for containers 2. Artifact collection support - 104690 <https://gerrit.ovirt.org/104690>: Added NFS server container image - 104273 <https://gerrit.ovirt.org/104273>: STDCI PODS: Unique UID for each job build's POD - 104756 <https://gerrit.ovirt.org/104756>: pipeline-loader: refactor: separate podspec func - 104757 <https://gerrit.ovirt.org/104757>: pipeline-loader: refactor: Use podspec struct def - 104766 <https://gerrit.ovirt.org/104766>: STDCI PODs: Add artifact collection logic - 105522 <https://gerrit.ovirt.org/105522>: Documented artifact collection in containers 3. Extended log collection - 104842 <https://gerrit.ovirt.org/104842>: STDCI PODs: Add POD log collection - 105523 <https://gerrit.ovirt.org/105523>: Documented log collection in containers 4. Privileged container support - 104786 <https://gerrit.ovirt.org/104786>: STDCI DSL: Enable privileged containers 5. Support for using containers in gating jobs: - 104804 <https://gerrit.ovirt.org/104804>: standard-stage: refactor: move DSL to a library - 104811 <https://gerrit.ovirt.org/104811>: gate: Support getting suits from STDCI DSL 6. Providing the `extra_sources` file to OST suit containers: - 104843 <https://gerrit.ovirt.org/104843>: stdci_runner: Create extra_sources for PODs 7. Support for mirror injection and upstream source cloning - 104917 <https://gerrit.ovirt.org/104917>: Added a container with STDCI tools - 104918 <https://gerrit.ovirt.org/104918>: decorate.py: Add script - 104989 <https://gerrit.ovirt.org/104989>: STDCI DSL: Use `tools` container for `decorate` - 104994 <https://gerrit.ovirt.org/104994>: stdci_runner: Inject mirrors in PODs As you can see, we have quite a big pile of reviews to do, as always, help is very welcome... Regards, Barak. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted

2 4

[oVirt Jenkins] ovirt-system-tests_compat-4.3-suite-master - Build # 290 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Jan '20

29 Jan '20

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/290/ Build Number: 290 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #290 [Marcin Sobczyk] deploy: Fix vdsm coverage script ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.verify_add_hosts_43 Error Message: None != True after 1200 seconds -------------------- >> begin captured logging << -------------------- 002_bootstrap: DEBUG: Host status, verify_add_hosts: lago-compat-4-3-suite-master-host-0: install_failed lago-compat-4-3-suite-master-host-1: installing --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 363, in verify_add_hosts timeout=constants.ADD_HOST_TIMEOUT File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) 'None != True after 1200 seconds\n-------------------- >> begin captured logging << --------------------\n002_bootstrap: DEBUG: Host status, verify_add_hosts:\n lago-compat-4-3-suite-master-host-0: install_failed\nlago-compat-4-3-suite-master-host-1: installing\n\n--------------------- >> end captured logging << ---------------------'

1 43

Build failed in Jenkins: system-sync_mirrors-epel-s390x-el7-s390x #109
by jenkins＠jenkins.phx.ovirt.org 03 Jan '20

03 Jan '20

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-s390x-el7-s390x/109/d…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-s390x-el7-s390x/ws/> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 454b5bc96ad293356da6af51df1f62b03ec0e823 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 454b5bc96ad293356da6af51df1f62b03ec0e823 Commit message: "pusher: Consider dest branch when checking if push is needed" > git rev-list --no-walk 454b5bc96ad293356da6af51df1f62b03ec0e823 # timeout=10 [system-sync_mirrors-epel-s390x-el7-s390x] $ /bin/bash -xe /tmp/jenkins5659741722053903631.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-s390x-el7 s390x jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror epel-s390x-el7 s390x jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror epel-s390x-el7 s390x jenkins/data/mirrors-reposync.conf + local repo_name=epel-s390x-el7 + local repo_archs=s390x + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs epel-s390x-el7 yum + local repo_name=epel-s390x-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/epel-s390x-el7 /var/www/html/repos/yum/epel-s390x-el7/base + check_yum_sync_needed epel-s390x-el7 s390x jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=epel-s390x-el7 + local repo_archs=s390x + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/epel-s390x-el7 ++ IFS=, ++ echo s390x + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync epel-s390x-el7 s390x jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=epel-s390x-el7 ++ local repo_arch=s390x ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=epel-s390x-el7 --arch=s390x --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/epel-s390x-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: failure: repodata/bc08248d5f12773d2cacabd95b08cca382c882312507ff751455fb3b07d8ab22-primary.sqlite.bz2 from epel-s390x-el7: [Errno 256] No more mirrors to try. http://download.sinenomine.net/clefos/epel7/repodata/bc08248d5f12773d2cacab…: [Errno 14] HTTP Error 404 - Not Found + reposync_out= Build step 'Execute shell' marked build as failure

1 24

oVirt infra daily report - unstable production jobs - 1056
by jenkins＠jenkins.phx.ovirt.org 31 Dec '19

31 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1056//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-updates-7.6-el7-x86_64 #208
by jenkins＠jenkins.phx.ovirt.org 31 Dec '19

31 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-7.6-el7-x86…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-7.6-el7-x86…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision beba1d2f2e3531c000852f54431c5432bc76f149 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f beba1d2f2e3531c000852f54431c5432bc76f149 Commit message: "decorate.py: Add script" > git rev-list --no-walk beba1d2f2e3531c000852f54431c5432bc76f149 # timeout=10 [system-sync_mirrors-centos-updates-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins5376853183813924811.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-updates-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-updates-7.6-el7 yum + local repo_name=centos-updates-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-updates-7.6-el7 /var/www/html/repos/yum/centos-updates-7.6-el7/base + check_yum_sync_needed centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-updates-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-updates-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-updates-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-updates-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-updates-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-7.6-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-7.6-el7' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[CQ]: 105993, 2 (ovirt-engine-api-model) failed "ovirt-master" system tests
by oVirt Jenkins 31 Dec '19

31 Dec '19

Change 105993,2 (ovirt-engine-api-model) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105993/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17876/

1 0

[CQ]: 6b88172 (ovirt-engine-sdk-go) failed "ovirt-4.3" system tests
by oVirt Jenkins 31 Dec '19

31 Dec '19

Change 6b88172 (ovirt-engine-sdk-go) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-engine-sdk-go/commit/6b8817275dc97a3b99354ed… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2732/

1 0

[CQ]: 6b88172 (ovirt-engine-sdk-go) failed "ovirt-master" system tests
by oVirt Jenkins 31 Dec '19

31 Dec '19

Change 6b88172 (ovirt-engine-sdk-go) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-engine-sdk-go/commit/6b8817275dc97a3b99354ed… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17872/

1 0

oVirt infra daily report - unstable production jobs - 1055
by jenkins＠jenkins.phx.ovirt.org 30 Dec '19

30 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1055//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64 #204
by jenkins＠jenkins.phx.ovirt.org 30 Dec '19

30 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 454b5bc96ad293356da6af51df1f62b03ec0e823 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 454b5bc96ad293356da6af51df1f62b03ec0e823 Commit message: "pusher: Consider dest branch when checking if push is needed" > git rev-list --no-walk 454b5bc96ad293356da6af51df1f62b03ec0e823 # timeout=10 [system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins139911091726115604.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-sclo-rh-release-7.6-el7 yum + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7 /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base + check_yum_sync_needed centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-sclo-rh-release-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-sclo-rh-release-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: failure: repodata/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2 from centos-sclo-rh-release-7.6-el7: [Errno 256] No more mirrors to try. http://vault.centos.org/7.6.1810/sclo/x86_64/rh/repodata/be198d32d786c933be…: [Errno 16] error setting timestamp on file /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2 from http://vault.centos.org/7.6.1810/sclo/x86_64/rh/repodata/be198d32d786c933be…, OSError: [Errno 2] No such file or directory: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[CQ]: 99250,77 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 29 Dec '19

29 Dec '19

Change 99250,77 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/99250/77 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17864/

1 0

[oVirt Jenkins] cleanup-gated-dummy-master - Build #9 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 29 Dec '19

29 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-dummy-master/9/ Build Name: #9 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[CQ]: 98907,62 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 29 Dec '19

29 Dec '19

Change 98907,62 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/98907/62 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17863/

1 0

oVirt infra daily report - unstable production jobs - 1054
by jenkins＠jenkins.phx.ovirt.org 29 Dec '19

29 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1054//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105920,5 (ovirt-imageio) failed "ovirt-master" system tests
by oVirt Jenkins 29 Dec '19

29 Dec '19

Change 105920,5 (ovirt-imageio) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105920/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17856/

1 0

[CQ]: 105477, 3 (ovirt-engine-metrics) failed "ovirt-master" system tests
by oVirt Jenkins 29 Dec '19

29 Dec '19

Change 105477,3 (ovirt-engine-metrics) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105477/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17853/

1 0

oVirt infra daily report - unstable production jobs - 1053
by jenkins＠jenkins.phx.ovirt.org 28 Dec '19

28 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1053//artifact/exported-… Cheers, Jenkins

1 0

[oVirt Jenkins] cleanup-gated-dummy-master - Build #7 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 27 Dec '19

27 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-dummy-master/7/ Build Name: #7 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1052
by jenkins＠jenkins.phx.ovirt.org 27 Dec '19

27 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1052//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64 #196
by jenkins＠jenkins.phx.ovirt.org 27 Dec '19

27 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 454b5bc96ad293356da6af51df1f62b03ec0e823 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 454b5bc96ad293356da6af51df1f62b03ec0e823 Commit message: "pusher: Consider dest branch when checking if push is needed" > git rev-list --no-walk 454b5bc96ad293356da6af51df1f62b03ec0e823 # timeout=10 [system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins1331483102574010344.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-sclo-rh-release-7.6-el7 yum + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7 /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base + check_yum_sync_needed centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-sclo-rh-release-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-sclo-rh-release-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: failure: repodata/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2 from centos-sclo-rh-release-7.6-el7: [Errno 256] No more mirrors to try. http://vault.centos.org/7.6.1810/sclo/x86_64/rh/repodata/be198d32d786c933be…: [Errno 16] error setting timestamp on file /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2 from http://vault.centos.org/7.6.1810/sclo/x86_64/rh/repodata/be198d32d786c933be…, OSError: [Errno 2] No such file or directory: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 1051
by jenkins＠jenkins.phx.ovirt.org 26 Dec '19

26 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1051//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105945,1 (ovirt-system-tests) failed "ovirt-4.3" system tests
by oVirt Jenkins 25 Dec '19

25 Dec '19

Change 105945,1 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105945/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2724/

1 0

[CQ]: 105945, 1 (ovirt-system-tests) failed "ovirt-master" system tests
by oVirt Jenkins 25 Dec '19

25 Dec '19

Change 105945,1 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105945/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17842/

1 0

oVirt infra daily report - unstable production jobs - 1050
by jenkins＠jenkins.phx.ovirt.org 25 Dec '19

25 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1050//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105932, 1 (ovirt-system-tests) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 24 Dec '19

24 Dec '19

A system test invoked by the "ovirt-4.3" change queue including change 105932,1 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 105785,1 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 105785,1 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/105932/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/105785/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2720/

1 0

[CQ]: 105932, 1 (ovirt-system-tests) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 24 Dec '19

24 Dec '19

A system test invoked by the "ovirt-master" change queue including change 105932,1 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 105785,1 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 105785,1 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/105932/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/105785/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17834/

1 0

[oVirt Jenkins] cleanup-gated-dummy-master - Build #3 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 24 Dec '19

24 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-dummy-master/3/ Build Name: #3 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[oVirt Jenkins] deploy-to-gated-ovirt-4.3 - Build #87 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 24 Dec '19

24 Dec '19

Build: http://jenkins.ovirt.org/job/deploy-to-gated-ovirt-4.3/87/ Build Name: #87 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 1

oVirt infra daily report - unstable production jobs - 1049
by jenkins＠jenkins.phx.ovirt.org 24 Dec '19

24 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1049//artifact/exported-… Cheers, Jenkins

1 0

OST check-patch takes a very long time
by Yedidyah Bar David 24 Dec '19

24 Dec '19

Hi all, Yesterday I rebased [1], and CI still didn't finish, after more than 24 hours. Checking the console log, I already see it's going to fail, at least due to a timeout: testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_standard-check-patch/ovirt-system-tests/he-node-ng-suite-4.3/test-scenarios/008_restart_he_vm.py", line 59, in restart_he_vm _wait_for_engine_health(host) File "/home/jenkins/agent/workspace/ovirt-system-tests_standard-check-patch/ovirt-system-tests/he-node-ng-suite-4.3/test-scenarios/008_restart_he_vm.py", line 191, in _wait_for_engine_health testlib.assert_true_within_long(lambda: any( File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) A previous rebase of the same patch also failed (after 4.5 hours), and another attempt (ci please test) failed after 33 hours. Does this make sense? Seems like there is very high load there for some reason. [1] https://gerrit.ovirt.org/104827 -- Didi

1 0

Move HE suite to centos8
by Yedidyah Bar David 24 Dec '19

24 Dec '19

Hi all, Galit has been working on [1] for some time now. Willing to help with that (and getting back to a stable state with hosted-engine suites), I looked at it a few days ago, after manual deploy on centos8 with nightly master snapshot worked for me. I rebased her patches, and CI failed [2]. Out of 5 suites that it tested, 3 passed (!), and 2 failed :-( . I checked the two that failed, and see: 1. One of them failed [3] due to failing to parse JSON output from 'hosted-engine --vm-status --json'. The output seems to have been cut in the middle, after 1024 bytes: 2019-12-22 11:13:47,846::ssh.py::ssh::58::lago.ssh::DEBUG::Running 200606d0 on lago-he-basic-role-remote-suite-4-3-host-0: hosted-engine --vm-status --json 2019-12-22 11:13:48,441::ssh.py::ssh::81::lago.ssh::DEBUG::Command 200606d0 on lago-he-basic-role-remote-suite-4-3-host-0 returned with 0 2019-12-22 11:13:48,441::ssh.py::ssh::89::lago.ssh::DEBUG::Command 200606d0 on lago-he-basic-role-remote-suite-4-3-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4534 (Sun Dec 22 06:13:39 2019)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4535 (Sun Dec 22 06:13:40 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-suite-4-3-host-0.lago.local", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "76bfd2ac", "local_conf_timestamp": 4535, "host-ts": 4534}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4532 (Sun Dec 22 06:13:38 2019)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4532 (Sun Dec 22 06:13:38 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-s 2019-12-22 11:13:48,442::testlib.py::assert_equals_within::242::ovirtlago.testlib::ERROR:: * Unhandled exception in <function <lambda> at 0x7fcc58f38668> Traceback (most recent call last): If you check previous runs of this command, in the same lago log, you see that they are complete (and somewhat longer, but not much), e.g.: 2019-12-22 11:13:41,804::ssh.py::ssh::58::lago.ssh::DEBUG::Running 1c6c1226 on lago-he-basic-role-remote-suite-4-3-host-0: hosted-engine --vm-status --json 2019-12-22 11:13:43,105::ssh.py::ssh::81::lago.ssh::DEBUG::Command 1c6c1226 on lago-he-basic-role-remote-suite-4-3-host-0 returned with 0 2019-12-22 11:13:43,106::ssh.py::ssh::89::lago.ssh::DEBUG::Command 1c6c1226 on lago-he-basic-role-remote-suite-4-3-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4534 (Sun Dec 22 06:13:39 2019)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4535 (Sun Dec 22 06:13:40 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-suite-4-3-host-0.lago.local", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "76bfd2ac", "local_conf_timestamp": 4535, "host-ts": 4534}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4532 (Sun Dec 22 06:13:38 2019)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4532 (Sun Dec 22 06:13:38 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-suite-4-3-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2fc395a1", "local_conf_timestamp": 4532, "host-ts": 4532}, "global_maintenance": true} So I suspect an infra issue - on the network level, ssh, etc. 2. The other seems to have failed [4] due to timeout: 2019-12-22 12:57:08,595::cmd.py::exit_handler::921::cli::DEBUG::signal 15 was caught The reason it took too long seems to be: 2019-12-22 11:08:13,233::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:57a44b4e-859e-4ff2-b897-7c8b3ade4557:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 11:08:13,637::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:57a44b4e-859e-4ff2-b897-7c8b3ade4557:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 11:08:13,942::ssh.py::ssh::58::lago.ssh::DEBUG::Running 59005d88 on lago-he-node-ng-suite-4-3-host-0: true 2019-12-22 11:08:15,605::ssh.py::ssh::81::lago.ssh::DEBUG::Command 59005d88 on lago-he-node-ng-suite-4-3-host-0 returned with 0 2019-12-22 11:08:15,606::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded for ssh to lago-he-node-ng-suite-4-3-host-0 2019-12-22 11:08:15,608::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:38e81fd0-50f0-497c-8fb0-38eedcfb3f55:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 11:08:16,129::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:38e81fd0-50f0-497c-8fb0-38eedcfb3f55:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 12:20:39,054::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:4007e002-f8e2-4427-884f-af57f723aca0:Get ssh client for lago-he-node-ng-suite-4-3-storage: 2019-12-22 12:20:39,639::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:4007e002-f8e2-4427-884f-af57f723aca0:Get ssh client for lago-he-node-ng-suite-4-3-storage: 2019-12-22 12:20:40,408::ssh.py::ssh::58::lago.ssh::DEBUG::Running 77b26f28 on lago-he-node-ng-suite-4-3-storage: true 2019-12-22 12:20:40,475::ssh.py::ssh::81::lago.ssh::DEBUG::Command 77b26f28 on lago-he-node-ng-suite-4-3-storage returned with 0 2019-12-22 12:20:40,476::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded for ssh to lago-he-node-ng-suite-4-3-storage Meaning, if I got it right, that it sshed to host-0, succeeded, then tried to ssh to storage, and also succeeded - but connecting took more than an hour. Again, I suspect some infra issue, perhaps same as (1.). Any clue? I am now trying it again, anyway [5]. Thanks and best regards, [1] https://gerrit.ovirt.org/104797 [2] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7269/ [3] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7269/… [4] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7269/… [5] http://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7273/ -- Didi

1 0

[oVirt Jenkins] cleanup-gated-ovirt-master - Build #9 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 23 Dec '19

23 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-ovirt-master/9/ Build Name: #9 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1048
by jenkins＠jenkins.phx.ovirt.org 23 Dec '19

23 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1048//artifact/exported-… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2850) Fedora 29 EOL
by Ehud Yonasi (oVirt JIRA) 23 Dec '19

23 Dec '19

Ehud Yonasi created OVIRT-2850: ---------------------------------- Summary: Fedora 29 EOL Key: OVIRT-2850 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2850 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Ehud Yonasi Assignee: infra We need to prepare patches to remove it + mirrors, and let developers know to remove fc29 builds from CI. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100118)

1 0

[oVirt Jenkins] cleanup-gated-ovirt-4.3 - Build #1 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 22 Dec '19

22 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-ovirt-4.3/1/ Build Name: #1 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 2

oVirt infra daily report - unstable production jobs - 1047
by jenkins＠jenkins.phx.ovirt.org 22 Dec '19

22 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1047//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-updates-7.6-el7-x86_64 #180
by jenkins＠jenkins.phx.ovirt.org 22 Dec '19

22 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-7.6-el7-x86…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-7.6-el7-x86…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 Commit message: "ovirt-engine-api-explorer: Add to STDCI v2" > git rev-list --no-walk 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 # timeout=10 [system-sync_mirrors-centos-updates-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins4808560237896478089.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-updates-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-updates-7.6-el7 yum + local repo_name=centos-updates-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-updates-7.6-el7 /var/www/html/repos/yum/centos-updates-7.6-el7/base + check_yum_sync_needed centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-updates-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-updates-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-updates-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-updates-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-updates-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: Error making cache directory: /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/gen error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/gen' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[CQ]: 105815,3 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 22 Dec '19

22 Dec '19

Change 105815,3 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105815/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17813/

1 0

oVirt infra daily report - unstable production jobs - 1046
by jenkins＠jenkins.phx.ovirt.org 21 Dec '19

21 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1046//artifact/exported-… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 1045
by jenkins＠jenkins.phx.ovirt.org 20 Dec '19

20 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1045//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64 #175
by jenkins＠jenkins.phx.ovirt.org 20 Dec '19

20 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 Commit message: "ovirt-engine-api-explorer: Add to STDCI v2" > git rev-list --no-walk 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 # timeout=10 [system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins2174981837918766167.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-sclo-rh-release-7.6-el7 yum + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7 /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base + check_yum_sync_needed centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-sclo-rh-release-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-sclo-rh-release-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: Error making cache directory: /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/packages error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/packages' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 1044
by jenkins＠jenkins.phx.ovirt.org 19 Dec '19

19 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1044//artifact/exported-… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2849) pusher should check both commit sha and branch when deciding if to push commit
by Emil Natan (oVirt JIRA) 19 Dec '19

19 Dec '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2849?page=com.atlassian.jira.… ] Emil Natan updated OVIRT-2849: ------------------------------ Description: When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both md5 hash and branch. (was: When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both commit id and branch. ) > pusher should check both commit sha and branch when deciding if to push commit > ------------------------------------------------------------------------------ > > Key: OVIRT-2849 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2849 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Emil Natan > Assignee: infra > Priority: High > > When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both md5 hash and branch. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100117)

1 0

[CQ]: 105802,1 (ovirt-appliance) failed "ovirt-master" system tests
by oVirt Jenkins 19 Dec '19

19 Dec '19

Change 105802,1 (ovirt-appliance) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105802/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17806/

1 0

[CQ]: 105811,4 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 18 Dec '19

18 Dec '19

Change 105811,4 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105811/4 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17803/

1 0

oVirt infra daily report - unstable production jobs - 1043
by jenkins＠jenkins.phx.ovirt.org 18 Dec '19

18 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1043//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105819,1 (vdsm) failed "ovirt-4.3" system tests
by oVirt Jenkins 18 Dec '19

18 Dec '19

Change 105819,1 (vdsm) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105819/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2713/

1 0

Build failed in Jenkins: system-sync_mirrors-sac-gluster-ansible-el7-x86_64 #449
by jenkins＠jenkins.phx.ovirt.org 18 Dec '19

18 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-sac-gluster-ansible-el7-x8…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-sac-gluster-ansible-el7-x8…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:894) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Failed to connect to 2600:1f18:26e2:3333::3333: Network is unreachable at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2172) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1864) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:78) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:545) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to mirrors.phx.ovirt.org at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:957) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor655.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy98.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:892) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) ERROR: Error fetching remote repo 'origin'

1 1

[JIRA] (OVIRT-2849) pusher should check both commit sha and branch when deciding if to push commit
by Emil Natan (oVirt JIRA) 18 Dec '19

18 Dec '19

Emil Natan created OVIRT-2849: --------------------------------- Summary: pusher should check both commit sha and branch when deciding if to push commit Key: OVIRT-2849 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2849 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Emil Natan Assignee: infra Priority: High When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both commit id and branch. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100117)

1 0

Re: Change in jenkins[master]: ovirt-engine-api-explorer: Add to STDCI v2
by Yedidyah Bar David 18 Dec '19

18 Dec '19

On Wed, Dec 18, 2019 at 12:23 PM Code Review <gerrit(a)ovirt.org> wrote: > > From Jenkins CI <jenkins(a)ovirt.org>: > > Jenkins CI has posted comments on this change. > > Change subject: ovirt-engine-api-explorer: Add to STDCI v2 > ...................................................................... > > > Patch Set 1: Continuous-Integration-1 > > Build Failed > > http://jenkins.ovirt.org/job/jenkins_standard-check-patch/4044/ : FAILURE java.lang.OutOfMemoryError: unable to create new native thread Any idea? > > http://jenkins.ovirt.org/job/jenkins_master_check-patch-el7-x86_64/7624/ : SUCCESS > > http://jenkins.ovirt.org/job/jenkins_master_check-patch-el7-ppc64le/4756/ : SUCCESS > > http://jenkins.ovirt.org/job/jenkins_master_check-patch-fc29-x86_64/2025/ : SUCCESS > > -- > To view, visit https://gerrit.ovirt.org/105806 > To unsubscribe, visit https://gerrit.ovirt.org/settings > > Gerrit-Project: jenkins > Gerrit-Branch: master > Gerrit-MessageType: comment > Gerrit-Change-Id: I1c15be302d28fd38b93ec8dcfd620dff7849f3be > Gerrit-Change-Number: 105806 > Gerrit-PatchSet: 1 > Gerrit-Owner: Yedidyah Bar David <didi(a)redhat.com> > Gerrit-Reviewer: Anton Marchukov <amarchuk(a)redhat.com> > Gerrit-Reviewer: Barak Korren <bkorren(a)redhat.com> > Gerrit-Reviewer: Dafna Ron <dron(a)redhat.com> > Gerrit-Reviewer: Daniel Belenky <dbelenky(a)redhat.com> > Gerrit-Reviewer: Ehud Yonasi <eyonasi(a)redhat.com> > Gerrit-Reviewer: Emil Natan <enatan(a)redhat.com> > Gerrit-Reviewer: Evgheni Dereveanchin <ederevea(a)redhat.com> > Gerrit-Reviewer: Eyal Edri <eedri(a)redhat.com> > Gerrit-Reviewer: Gal Ben Haim <galbh2(a)gmail.com> > Gerrit-Reviewer: Galit Rosenthal <grosenth(a)redhat.com> > Gerrit-Reviewer: Jenkins CI <jenkins(a)ovirt.org> > Gerrit-Reviewer: Sandro Bonazzola <sbonazzo(a)redhat.com> > Gerrit-Reviewer: Staging Jenkins Server (1001567) > Gerrit-Reviewer: Yedidyah Bar David <didi(a)redhat.com> > Gerrit-Comment-Date: Wed, 18 Dec 2019 10:23:08 +0000 > Gerrit-HasComments: No > -- Didi

1 0