- Infra - oVirt List Archives

[CQ]: 105920,5 (ovirt-imageio) failed "ovirt-master" system tests
by oVirt Jenkins 29 Dec '19

29 Dec '19

Change 105920,5 (ovirt-imageio) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105920/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17856/

1 0

[CQ]: 105477, 3 (ovirt-engine-metrics) failed "ovirt-master" system tests
by oVirt Jenkins 29 Dec '19

29 Dec '19

Change 105477,3 (ovirt-engine-metrics) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105477/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17853/

1 0

oVirt infra daily report - unstable production jobs - 1053
by jenkins＠jenkins.phx.ovirt.org 28 Dec '19

28 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1053//artifact/exported-… Cheers, Jenkins

1 0

[oVirt Jenkins] cleanup-gated-dummy-master - Build #7 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 27 Dec '19

27 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-dummy-master/7/ Build Name: #7 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1052
by jenkins＠jenkins.phx.ovirt.org 27 Dec '19

27 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1052//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64 #196
by jenkins＠jenkins.phx.ovirt.org 27 Dec '19

27 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 454b5bc96ad293356da6af51df1f62b03ec0e823 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 454b5bc96ad293356da6af51df1f62b03ec0e823 Commit message: "pusher: Consider dest branch when checking if push is needed" > git rev-list --no-walk 454b5bc96ad293356da6af51df1f62b03ec0e823 # timeout=10 [system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins1331483102574010344.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-sclo-rh-release-7.6-el7 yum + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7 /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base + check_yum_sync_needed centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-sclo-rh-release-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-sclo-rh-release-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: failure: repodata/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2 from centos-sclo-rh-release-7.6-el7: [Errno 256] No more mirrors to try. http://vault.centos.org/7.6.1810/sclo/x86_64/rh/repodata/be198d32d786c933be…: [Errno 16] error setting timestamp on file /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2 from http://vault.centos.org/7.6.1810/sclo/x86_64/rh/repodata/be198d32d786c933be…, OSError: [Errno 2] No such file or directory: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/be198d32d786c933beb06e141f7f49d7b9671df70ecb09f05c39370d498a5f57-primary.sqlite.bz2' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 1051
by jenkins＠jenkins.phx.ovirt.org 26 Dec '19

26 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1051//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105945,1 (ovirt-system-tests) failed "ovirt-4.3" system tests
by oVirt Jenkins 25 Dec '19

25 Dec '19

Change 105945,1 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105945/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2724/

1 0

[CQ]: 105945, 1 (ovirt-system-tests) failed "ovirt-master" system tests
by oVirt Jenkins 25 Dec '19

25 Dec '19

Change 105945,1 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105945/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17842/

1 0

oVirt infra daily report - unstable production jobs - 1050
by jenkins＠jenkins.phx.ovirt.org 25 Dec '19

25 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1050//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105932, 1 (ovirt-system-tests) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 24 Dec '19

24 Dec '19

A system test invoked by the "ovirt-4.3" change queue including change 105932,1 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 105785,1 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 105785,1 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/105932/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/105785/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2720/

1 0

[CQ]: 105932, 1 (ovirt-system-tests) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 24 Dec '19

24 Dec '19

A system test invoked by the "ovirt-master" change queue including change 105932,1 (ovirt-system-tests) failed. However, this change seems not to be the root cause for this failure. Change 105785,1 (ovirt-system-tests) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 105785,1 (ovirt-system-tests) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/105932/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/105785/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17834/

1 0

[oVirt Jenkins] cleanup-gated-dummy-master - Build #3 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 24 Dec '19

24 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-dummy-master/3/ Build Name: #3 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

[oVirt Jenkins] deploy-to-gated-ovirt-4.3 - Build #87 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 24 Dec '19

24 Dec '19

Build: http://jenkins.ovirt.org/job/deploy-to-gated-ovirt-4.3/87/ Build Name: #87 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 1

oVirt infra daily report - unstable production jobs - 1049
by jenkins＠jenkins.phx.ovirt.org 24 Dec '19

24 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1049//artifact/exported-… Cheers, Jenkins

1 0

OST check-patch takes a very long time
by Yedidyah Bar David 24 Dec '19

24 Dec '19

Hi all, Yesterday I rebased [1], and CI still didn't finish, after more than 24 hours. Checking the console log, I already see it's going to fail, at least due to a timeout: testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/agent/workspace/ovirt-system-tests_standard-check-patch/ovirt-system-tests/he-node-ng-suite-4.3/test-scenarios/008_restart_he_vm.py", line 59, in restart_he_vm _wait_for_engine_health(host) File "/home/jenkins/agent/workspace/ovirt-system-tests_standard-check-patch/ovirt-system-tests/he-node-ng-suite-4.3/test-scenarios/008_restart_he_vm.py", line 191, in _wait_for_engine_health testlib.assert_true_within_long(lambda: any( File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) A previous rebase of the same patch also failed (after 4.5 hours), and another attempt (ci please test) failed after 33 hours. Does this make sense? Seems like there is very high load there for some reason. [1] https://gerrit.ovirt.org/104827 -- Didi

1 0

Move HE suite to centos8
by Yedidyah Bar David 24 Dec '19

24 Dec '19

Hi all, Galit has been working on [1] for some time now. Willing to help with that (and getting back to a stable state with hosted-engine suites), I looked at it a few days ago, after manual deploy on centos8 with nightly master snapshot worked for me. I rebased her patches, and CI failed [2]. Out of 5 suites that it tested, 3 passed (!), and 2 failed :-( . I checked the two that failed, and see: 1. One of them failed [3] due to failing to parse JSON output from 'hosted-engine --vm-status --json'. The output seems to have been cut in the middle, after 1024 bytes: 2019-12-22 11:13:47,846::ssh.py::ssh::58::lago.ssh::DEBUG::Running 200606d0 on lago-he-basic-role-remote-suite-4-3-host-0: hosted-engine --vm-status --json 2019-12-22 11:13:48,441::ssh.py::ssh::81::lago.ssh::DEBUG::Command 200606d0 on lago-he-basic-role-remote-suite-4-3-host-0 returned with 0 2019-12-22 11:13:48,441::ssh.py::ssh::89::lago.ssh::DEBUG::Command 200606d0 on lago-he-basic-role-remote-suite-4-3-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4534 (Sun Dec 22 06:13:39 2019)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4535 (Sun Dec 22 06:13:40 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-suite-4-3-host-0.lago.local", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "76bfd2ac", "local_conf_timestamp": 4535, "host-ts": 4534}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4532 (Sun Dec 22 06:13:38 2019)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4532 (Sun Dec 22 06:13:38 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-s 2019-12-22 11:13:48,442::testlib.py::assert_equals_within::242::ovirtlago.testlib::ERROR:: * Unhandled exception in <function <lambda> at 0x7fcc58f38668> Traceback (most recent call last): If you check previous runs of this command, in the same lago log, you see that they are complete (and somewhat longer, but not much), e.g.: 2019-12-22 11:13:41,804::ssh.py::ssh::58::lago.ssh::DEBUG::Running 1c6c1226 on lago-he-basic-role-remote-suite-4-3-host-0: hosted-engine --vm-status --json 2019-12-22 11:13:43,105::ssh.py::ssh::81::lago.ssh::DEBUG::Command 1c6c1226 on lago-he-basic-role-remote-suite-4-3-host-0 returned with 0 2019-12-22 11:13:43,106::ssh.py::ssh::89::lago.ssh::DEBUG::Command 1c6c1226 on lago-he-basic-role-remote-suite-4-3-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4534 (Sun Dec 22 06:13:39 2019)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=4535 (Sun Dec 22 06:13:40 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-suite-4-3-host-0.lago.local", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "76bfd2ac", "local_conf_timestamp": 4535, "host-ts": 4534}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4532 (Sun Dec 22 06:13:38 2019)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4532 (Sun Dec 22 06:13:38 2019)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-role-remote-suite-4-3-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "2fc395a1", "local_conf_timestamp": 4532, "host-ts": 4532}, "global_maintenance": true} So I suspect an infra issue - on the network level, ssh, etc. 2. The other seems to have failed [4] due to timeout: 2019-12-22 12:57:08,595::cmd.py::exit_handler::921::cli::DEBUG::signal 15 was caught The reason it took too long seems to be: 2019-12-22 11:08:13,233::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:57a44b4e-859e-4ff2-b897-7c8b3ade4557:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 11:08:13,637::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:57a44b4e-859e-4ff2-b897-7c8b3ade4557:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 11:08:13,942::ssh.py::ssh::58::lago.ssh::DEBUG::Running 59005d88 on lago-he-node-ng-suite-4-3-host-0: true 2019-12-22 11:08:15,605::ssh.py::ssh::81::lago.ssh::DEBUG::Command 59005d88 on lago-he-node-ng-suite-4-3-host-0 returned with 0 2019-12-22 11:08:15,606::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded for ssh to lago-he-node-ng-suite-4-3-host-0 2019-12-22 11:08:15,608::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:38e81fd0-50f0-497c-8fb0-38eedcfb3f55:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 11:08:16,129::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:38e81fd0-50f0-497c-8fb0-38eedcfb3f55:Get ssh client for lago-he-node-ng-suite-4-3-host-0: 2019-12-22 12:20:39,054::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:4007e002-f8e2-4427-884f-af57f723aca0:Get ssh client for lago-he-node-ng-suite-4-3-storage: 2019-12-22 12:20:39,639::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:4007e002-f8e2-4427-884f-af57f723aca0:Get ssh client for lago-he-node-ng-suite-4-3-storage: 2019-12-22 12:20:40,408::ssh.py::ssh::58::lago.ssh::DEBUG::Running 77b26f28 on lago-he-node-ng-suite-4-3-storage: true 2019-12-22 12:20:40,475::ssh.py::ssh::81::lago.ssh::DEBUG::Command 77b26f28 on lago-he-node-ng-suite-4-3-storage returned with 0 2019-12-22 12:20:40,476::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded for ssh to lago-he-node-ng-suite-4-3-storage Meaning, if I got it right, that it sshed to host-0, succeeded, then tried to ssh to storage, and also succeeded - but connecting took more than an hour. Again, I suspect some infra issue, perhaps same as (1.). Any clue? I am now trying it again, anyway [5]. Thanks and best regards, [1] https://gerrit.ovirt.org/104797 [2] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7269/ [3] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7269/… [4] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7269/… [5] http://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7273/ -- Didi

1 0

[oVirt Jenkins] cleanup-gated-ovirt-master - Build #9 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 23 Dec '19

23 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-ovirt-master/9/ Build Name: #9 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 0

oVirt infra daily report - unstable production jobs - 1048
by jenkins＠jenkins.phx.ovirt.org 23 Dec '19

23 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1048//artifact/exported-… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2850) Fedora 29 EOL
by Ehud Yonasi (oVirt JIRA) 23 Dec '19

23 Dec '19

Ehud Yonasi created OVIRT-2850: ---------------------------------- Summary: Fedora 29 EOL Key: OVIRT-2850 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2850 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Ehud Yonasi Assignee: infra We need to prepare patches to remove it + mirrors, and let developers know to remove fc29 builds from CI. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100118)

1 0

[oVirt Jenkins] cleanup-gated-ovirt-4.3 - Build #1 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 22 Dec '19

22 Dec '19

Build: http://jenkins.ovirt.org/job/cleanup-gated-ovirt-4.3/1/ Build Name: #1 Build Status: FAILURE Gerrit change: null - title: null - project: null - branch: null - author: null <null>

1 2

oVirt infra daily report - unstable production jobs - 1047
by jenkins＠jenkins.phx.ovirt.org 22 Dec '19

22 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1047//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-updates-7.6-el7-x86_64 #180
by jenkins＠jenkins.phx.ovirt.org 22 Dec '19

22 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-7.6-el7-x86…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-7.6-el7-x86…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 Commit message: "ovirt-engine-api-explorer: Add to STDCI v2" > git rev-list --no-walk 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 # timeout=10 [system-sync_mirrors-centos-updates-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins4808560237896478089.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-updates-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-updates-7.6-el7 yum + local repo_name=centos-updates-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-updates-7.6-el7 /var/www/html/repos/yum/centos-updates-7.6-el7/base + check_yum_sync_needed centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-updates-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-updates-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-updates-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-updates-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-updates-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-updates-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: Error making cache directory: /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/gen error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/gen' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[CQ]: 105815,3 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 22 Dec '19

22 Dec '19

Change 105815,3 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105815/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17813/

1 0

oVirt infra daily report - unstable production jobs - 1046
by jenkins＠jenkins.phx.ovirt.org 21 Dec '19

21 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1046//artifact/exported-… Cheers, Jenkins

1 0

oVirt infra daily report - unstable production jobs - 1045
by jenkins＠jenkins.phx.ovirt.org 20 Dec '19

20 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1045//artifact/exported-… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64 #175
by jenkins＠jenkins.phx.ovirt.org 20 Dec '19

20 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7.6…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 Commit message: "ovirt-engine-api-explorer: Add to STDCI v2" > git rev-list --no-walk 9bb80cca44ae31e3f7c9f48c43f3386abb445d34 # timeout=10 [system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins2174981837918766167.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs centos-sclo-rh-release-7.6-el7 yum + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7 /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base + check_yum_sync_needed centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=centos-sclo-rh-release-7.6-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=centos-sclo-rh-release-7.6-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-sclo-rh-release-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base --norepopath --newest-only --urls --quiet Error setting up repositories: Error making cache directory: /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/packages error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7/packages' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 1044
by jenkins＠jenkins.phx.ovirt.org 19 Dec '19

19 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1044//artifact/exported-… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2849) pusher should check both commit sha and branch when deciding if to push commit
by Emil Natan (oVirt JIRA) 19 Dec '19

19 Dec '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2849?page=com.atlassian.jira.… ] Emil Natan updated OVIRT-2849: ------------------------------ Description: When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both md5 hash and branch. (was: When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both commit id and branch. ) > pusher should check both commit sha and branch when deciding if to push commit > ------------------------------------------------------------------------------ > > Key: OVIRT-2849 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2849 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Emil Natan > Assignee: infra > Priority: High > > When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both md5 hash and branch. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100117)

1 0

[CQ]: 105802,1 (ovirt-appliance) failed "ovirt-master" system tests
by oVirt Jenkins 19 Dec '19

19 Dec '19

Change 105802,1 (ovirt-appliance) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105802/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17806/

1 0

[CQ]: 105811,4 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 18 Dec '19

18 Dec '19

Change 105811,4 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105811/4 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17803/

1 0

oVirt infra daily report - unstable production jobs - 1043
by jenkins＠jenkins.phx.ovirt.org 18 Dec '19

18 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1043//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105819,1 (vdsm) failed "ovirt-4.3" system tests
by oVirt Jenkins 18 Dec '19

18 Dec '19

Change 105819,1 (vdsm) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105819/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2713/

1 0

Build failed in Jenkins: system-sync_mirrors-sac-gluster-ansible-el7-x86_64 #449
by jenkins＠jenkins.phx.ovirt.org 18 Dec '19

18 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-sac-gluster-ansible-el7-x8…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-sac-gluster-ansible-el7-x8…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:894) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Failed to connect to 2600:1f18:26e2:3333::3333: Network is unreachable at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2172) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1864) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:78) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:545) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to mirrors.phx.ovirt.org at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:957) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor655.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy98.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:892) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) ERROR: Error fetching remote repo 'origin'

1 1

[JIRA] (OVIRT-2849) pusher should check both commit sha and branch when deciding if to push commit
by Emil Natan (oVirt JIRA) 18 Dec '19

18 Dec '19

Emil Natan created OVIRT-2849: --------------------------------- Summary: pusher should check both commit sha and branch when deciding if to push commit Key: OVIRT-2849 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2849 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Emil Natan Assignee: infra Priority: High When pushing the same upstream commit to multiple branches, once the commit is pushed to one branch, this will prevent the same commit to be pushed to other branches. Verification if to push commit should take in account both commit id and branch. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100117)

1 0

Re: Change in jenkins[master]: ovirt-engine-api-explorer: Add to STDCI v2
by Yedidyah Bar David 18 Dec '19

18 Dec '19

On Wed, Dec 18, 2019 at 12:23 PM Code Review <gerrit(a)ovirt.org> wrote: > > From Jenkins CI <jenkins(a)ovirt.org>: > > Jenkins CI has posted comments on this change. > > Change subject: ovirt-engine-api-explorer: Add to STDCI v2 > ...................................................................... > > > Patch Set 1: Continuous-Integration-1 > > Build Failed > > http://jenkins.ovirt.org/job/jenkins_standard-check-patch/4044/ : FAILURE java.lang.OutOfMemoryError: unable to create new native thread Any idea? > > http://jenkins.ovirt.org/job/jenkins_master_check-patch-el7-x86_64/7624/ : SUCCESS > > http://jenkins.ovirt.org/job/jenkins_master_check-patch-el7-ppc64le/4756/ : SUCCESS > > http://jenkins.ovirt.org/job/jenkins_master_check-patch-fc29-x86_64/2025/ : SUCCESS > > -- > To view, visit https://gerrit.ovirt.org/105806 > To unsubscribe, visit https://gerrit.ovirt.org/settings > > Gerrit-Project: jenkins > Gerrit-Branch: master > Gerrit-MessageType: comment > Gerrit-Change-Id: I1c15be302d28fd38b93ec8dcfd620dff7849f3be > Gerrit-Change-Number: 105806 > Gerrit-PatchSet: 1 > Gerrit-Owner: Yedidyah Bar David <didi(a)redhat.com> > Gerrit-Reviewer: Anton Marchukov <amarchuk(a)redhat.com> > Gerrit-Reviewer: Barak Korren <bkorren(a)redhat.com> > Gerrit-Reviewer: Dafna Ron <dron(a)redhat.com> > Gerrit-Reviewer: Daniel Belenky <dbelenky(a)redhat.com> > Gerrit-Reviewer: Ehud Yonasi <eyonasi(a)redhat.com> > Gerrit-Reviewer: Emil Natan <enatan(a)redhat.com> > Gerrit-Reviewer: Evgheni Dereveanchin <ederevea(a)redhat.com> > Gerrit-Reviewer: Eyal Edri <eedri(a)redhat.com> > Gerrit-Reviewer: Gal Ben Haim <galbh2(a)gmail.com> > Gerrit-Reviewer: Galit Rosenthal <grosenth(a)redhat.com> > Gerrit-Reviewer: Jenkins CI <jenkins(a)ovirt.org> > Gerrit-Reviewer: Sandro Bonazzola <sbonazzo(a)redhat.com> > Gerrit-Reviewer: Staging Jenkins Server (1001567) > Gerrit-Reviewer: Yedidyah Bar David <didi(a)redhat.com> > Gerrit-Comment-Date: Wed, 18 Dec 2019 10:23:08 +0000 > Gerrit-HasComments: No > -- Didi

1 0

Build failed in Jenkins: system-sync_mirrors-centos-qemu-ev-testing-el7-x86_64 #1931
by jenkins＠jenkins.phx.ovirt.org 18 Dec '19

18 Dec '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-qemu-ev-testing-el7…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-qemu-ev-testing-el7…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from http://gerrit.ovirt.org/jenkins.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:894) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'http://gerrit.ovirt.org/jenkins.git/': Failed to connect to 2600:1f18:26e2:3333::3333: Network is unreachable at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2172) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1864) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:78) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:545) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to mirrors.phx.ovirt.org at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:957) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor655.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy98.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:892) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) ERROR: Error fetching remote repo 'origin'

1 1

[CQ]: 105785, 1 (ovirt-system-tests) failed "ovirt-master" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 105785,1 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105785/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17792/

1 0

[CQ]: 105716,2 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 105716,2 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105716/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17786/

1 0

oVirt infra daily report - unstable production jobs - 1042
by jenkins＠jenkins.phx.ovirt.org 17 Dec '19

17 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1042//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105184,6 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 105184,6 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105184/6 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17775/

1 0

[CQ]: 105785,1 (ovirt-system-tests) failed "ovirt-4.3" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 105785,1 (ovirt-system-tests) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105785/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2704/

1 0

[CQ]: 105777,5 (ovirt-provider-ovn) failed "ovirt-4.3" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 105777,5 (ovirt-provider-ovn) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/105777/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2701/

1 0

[CQ]: 102665,15 (jenkins) failed "dummy-master" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 102665,15 (jenkins) is probably the reason behind recent system test failures in the "dummy-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/102665/15 For failed test results see: https://jenkins-staging.ovirt.org/job/dummy-master_change-queue-tester/472/

1 0

CI system outage
by Evgheni Dereveanchin 17 Dec '19

17 Dec '19

Hello everyone, We received several reports of Jenkins jobs not running properly. The reason turned out to be the Jenkins Master which did not start jobs properly. It has been restarted and is working properly now. If you have any jobs that failed or did not start at all today - please re-trigger them or reach out to me for assistance. -- Regards, Evgheni Dereveanchin

1 0

[CQ]: 104756,8 (jenkins) failed "dummy-master" system tests
by oVirt Jenkins 17 Dec '19

17 Dec '19

Change 104756,8 (jenkins) is probably the reason behind recent system test failures in the "dummy-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/104756/8 For failed test results see: http://jenkins.ovirt.org/job/dummy-master_change-queue-tester/240/

1 0

published master engine is 13 days old
by Yedidyah Bar David 17 Dec '19

17 Dec '19

Hi all, $subject. [1] has ovirt-engine-4.4.0-0.0.master.20191204120550.git04d5d05.el7.noarch . Tried to look around, and I have a few notes/questions: 1. Last successful run of [2] is 3 days old, but apparently it wasn't published. Any idea why? 2. Failed runs of [2] are reported to infra, with emails such as: [CQ]: 105472, 5 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause Is anyone monitoring these? Is this the only alerting that CI generates on such failures? If first is No and second is Yes, then we need someone/something to start monitoring. This was discussed a lot, but I do not see any change. Ideally, such alerts should be To'ed or Cc'ed to the author and reviewers of the patch that CI found to be guilty (which might be wrong, that's not the point). Do we plan to have something like this? Any idea when it will be ready? 3. I looked at a few recent failures of [2], specifically [3][4]. Both seem to have been killed after a timeout, while running 'engine-config'. For [3] that's clear, see [5]: 2019-12-16 17:11:44,766::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:fb6611dc-55bb-4251-aeda-2578b2ec83a2:Get ssh client for lago-basic-suite-master-engine: 2019-12-16 17:11:44,931::ssh.py::ssh::58::lago.ssh::DEBUG::Running 22e2b6b6 on lago-basic-suite-master-engine: engine-config --set VdsmUseNmstate=true 2019-12-16 19:55:21,965::cmd.py::exit_handler::921::cli::DEBUG::signal 15 was caught Can't find stdout/stderr of engine-config, so it's hard to tell if it outputted anything helpful to understand why it was stuck. It's hard to tell that about [4], because it has very few artifacts collected, no idea why, notably no lago.log, but [6] does show: [36m # initialize_engine: [32mSuccess [0m (in 0:04:00) [0m [36m # engine_config: [0m [0m [0m [36m * Collect artifacts: [0m [0m [0m [36m - [Thread-34] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:04) [0m [36m * Collect artifacts: [31mERROR [0m (in 0:00:04) [0m [36m # engine_config: [31mERROR [0m (in 2:42:57) [0m /bin/bash: line 31: 5225 Killed ${_STDCI_TIMEOUT_CMD} "3h" "$script_path" < /dev/null If I run 'engine-config --set VdsmUseNmstate=true' on my 20191204120550.git04d5d05 engine, it returns quickly. Tried also adding a repo pointing at last successful run of [7], which is currently [8], and it prompts me to input a version, probably as a result of [9]. Ales/Martin, can you please have a look? Thanks. [1] https://resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7/noarch/ [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/ [3] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17768/ [4] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17761/ [5] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17768/artifa… [6] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17761/artifa… [7] https://jenkins.ovirt.org/job/ovirt-engine_standard-on-merge/ [8] https://jenkins.ovirt.org/job/ovirt-engine_standard-on-merge/384/ [9] https://gerrit.ovirt.org/105440 -- Didi

3 3

oVirt infra daily report - unstable production jobs - 1041
by jenkins＠jenkins.phx.ovirt.org 16 Dec '19

16 Dec '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1041//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 105472, 5 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 16 Dec '19

16 Dec '19

A system test invoked by the "ovirt-master" change queue including change 105472,5 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 104569,2 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 104569,2 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/105472/5 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/104569/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/17768/

1 0

[JIRA] (OVIRT-2848) Ability to upload to artifactory.orvit.org
by Evgheni Dereveanchin (oVirt JIRA) 16 Dec '19

16 Dec '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2848?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-2848: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > Ability to upload to artifactory.orvit.org > ------------------------------------------ > > Key: OVIRT-2848 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2848 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Piotr Kliczewski > Assignee: Evgheni Dereveanchin > > Hi, > I would like to be able to upload to artifactory because I am experiencing > issues with maven central. > Thanks, > Piotr -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100116)

1 0