- Infra - oVirt List Archives

[CQ]: b232015 (ovirt-ansible-hosted-engine-setup) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 01 Aug '19

01 Aug '19

A system test invoked by the "ovirt-4.2" change queue including change b232015 (ovirt-ansible-hosted-engine-setup) failed. However, this change seems not to be the root cause for this failure. Change e68aa26 (ovirt-ansible-hosted- engine-setup) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change e68aa26 (ovirt-ansible-hosted- engine-setup) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/b2320151f… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/e68aa26a2… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/4673/

1 0

[CQ]: b232015 (ovirt-ansible-hosted-engine-setup) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 01 Aug '19

01 Aug '19

A system test invoked by the "ovirt-master" change queue including change b232015 (ovirt-ansible-hosted-engine-setup) failed. However, this change seems not to be the root cause for this failure. Change e68aa26 (ovirt-ansible- hosted-engine-setup) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change e68aa26 (ovirt-ansible-hosted- engine-setup) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/b2320151f… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/e68aa26a2… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15229/

1 0

[CQ]: 97c544b (ovirt-ansible-hosted-engine-setup) failed "ovirt-4.2" system tests, but isn't the failure root cause
by oVirt Jenkins 01 Aug '19

01 Aug '19

A system test invoked by the "ovirt-4.2" change queue including change 97c544b (ovirt-ansible-hosted-engine-setup) failed. However, this change seems not to be the root cause for this failure. Change e68aa26 (ovirt-ansible-hosted- engine-setup) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change e68aa26 (ovirt-ansible-hosted- engine-setup) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/97c544bca… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/e68aa26a2… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/4672/

1 0

[CQ]: 97c544b (ovirt-ansible-hosted-engine-setup) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 01 Aug '19

01 Aug '19

A system test invoked by the "ovirt-master" change queue including change 97c544b (ovirt-ansible-hosted-engine-setup) failed. However, this change seems not to be the root cause for this failure. Change e68aa26 (ovirt-ansible- hosted-engine-setup) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change e68aa26 (ovirt-ansible-hosted- engine-setup) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/97c544bca… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/e68aa26a2… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15228/

1 0

[JIRA] (OVIRT-2768) All loader-container nodes are offline, 99 jobs in the queue, jobs stuck for 2 hours
by Eyal Edri (oVirt JIRA) 01 Aug '19

01 Aug '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2768?page=com.atlassian.jira.… ] Eyal Edri commented on OVIRT-2768: ---------------------------------- This was caused by a bug in Jenkins that caused the thread that listens to JNLP to die. Evgheni, can you give some info on how it was fixed? On Wed, Jul 31, 2019 at 6:36 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > We depend on jenkins to run our tests and build artifacts in a timely > manner. > > Here is example job, stuck for 2 hours: > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/9212/ > > This looks like the issue: > > [image: collapse] > <https://jenkins.ovirt.org/toggleCollapse?paneId=executors>Build Executor > Status <https://jenkins.ovirt.org/computer/> > master <https://jenkins.ovirt.org/computer/(master)/> > 1 Idle > 2 Idle > 3 Idle > 4 Idle > lfedora1.lf-dev.marist.edu > <https://jenkins.ovirt.org/computer/lfedora1.lf-dev.marist.edu/> > 1 Idle > loader-container-28ljp > <https://jenkins.ovirt.org/computer/loader-container-28ljp/> (offline) > (suspended) > loader-container-35tlc > <https://jenkins.ovirt.org/computer/loader-container-35tlc/> (offline) > (suspended) > loader-container-3j0ps > <https://jenkins.ovirt.org/computer/loader-container-3j0ps/> (offline) > (suspended) > loader-container-4391q > <https://jenkins.ovirt.org/computer/loader-container-4391q/> (offline) > (suspended) > loader-container-58v3d > <https://jenkins.ovirt.org/computer/loader-container-58v3d/> (offline) > (suspended) > loader-container-66r5r > <https://jenkins.ovirt.org/computer/loader-container-66r5r/> (offline) > (suspended) > loader-container-6k57j > <https://jenkins.ovirt.org/computer/loader-container-6k57j/> (offline) > (suspended) > loader-container-7813f > <https://jenkins.ovirt.org/computer/loader-container-7813f/> (offline) > (suspended) > loader-container-8dkj1 > <https://jenkins.ovirt.org/computer/loader-container-8dkj1/> (offline) > (suspended) > loader-container-8hb98 > <https://jenkins.ovirt.org/computer/loader-container-8hb98/> (offline) > (suspended) > loader-container-8v609 > <https://jenkins.ovirt.org/computer/loader-container-8v609/> (offline) > (suspended) > loader-container-99f36 > <https://jenkins.ovirt.org/computer/loader-container-99f36/> (offline) > (suspended) > loader-container-c16rq > <https://jenkins.ovirt.org/computer/loader-container-c16rq/> (offline) > (suspended) > loader-container-c1vkj > <https://jenkins.ovirt.org/computer/loader-container-c1vkj/> (offline) > (suspended) > loader-container-c4g8c > <https://jenkins.ovirt.org/computer/loader-container-c4g8c/> (offline) > (suspended) > loader-container-cqrx5 > <https://jenkins.ovirt.org/computer/loader-container-cqrx5/> (offline) > (suspended) > loader-container-d1r8d > <https://jenkins.ovirt.org/computer/loader-container-d1r8d/> (offline) > (suspended) > loader-container-dpp70 > <https://jenkins.ovirt.org/computer/loader-container-dpp70/> (offline) > (suspended) > loader-container-gz52q > <https://jenkins.ovirt.org/computer/loader-container-gz52q/> (offline) > (suspended) > loader-container-h9kcf > <https://jenkins.ovirt.org/computer/loader-container-h9kcf/> (offline) > (suspended) > loader-container-hjq0g > <https://jenkins.ovirt.org/computer/loader-container-hjq0g/> (offline) > (suspended) > loader-container-ht75p > <https://jenkins.ovirt.org/computer/loader-container-ht75p/> (offline) > (suspended) > loader-container-htstk > <https://jenkins.ovirt.org/computer/loader-container-htstk/> (offline) > (suspended) > loader-container-j47s3 > <https://jenkins.ovirt.org/computer/loader-container-j47s3/> (offline) > (suspended) > loader-container-k2p10 > <https://jenkins.ovirt.org/computer/loader-container-k2p10/> (offline) > (suspended) > loader-container-kc066 > <https://jenkins.ovirt.org/computer/loader-container-kc066/> (offline) > (suspended) > loader-container-l1vd9 > <https://jenkins.ovirt.org/computer/loader-container-l1vd9/> (offline) > (suspended) > loader-container-l7lqn > <https://jenkins.ovirt.org/computer/loader-container-l7lqn/> (offline) > (suspended) > loader-container-lqqkj > <https://jenkins.ovirt.org/computer/loader-container-lqqkj/> (offline) > (suspended) > loader-container-m8fkq > <https://jenkins.ovirt.org/computer/loader-container-m8fkq/> (offline) > (suspended) > loader-container-n2zml > <https://jenkins.ovirt.org/computer/loader-container-n2zml/> (offline) > (suspended) > loader-container-p7rxw > <https://jenkins.ovirt.org/computer/loader-container-p7rxw/> (offline) > (suspended) > loader-container-pqg56 > <https://jenkins.ovirt.org/computer/loader-container-pqg56/> (offline) > (suspended) > loader-container-qtsdr > <https://jenkins.ovirt.org/computer/loader-container-qtsdr/> (offline) > (suspended) > loader-container-qxwnp > <https://jenkins.ovirt.org/computer/loader-container-qxwnp/> (offline) > (suspended) > loader-container-rdlww > <https://jenkins.ovirt.org/computer/loader-container-rdlww/> (offline) > (suspended) > loader-container-rp1rc > <https://jenkins.ovirt.org/computer/loader-container-rp1rc/> (offline) > (suspended) > loader-container-s6ndk > <https://jenkins.ovirt.org/computer/loader-container-s6ndk/> (offline) > (suspended) > loader-container-sv0pt > <https://jenkins.ovirt.org/computer/loader-container-sv0pt/> (offline) > (suspended) > loader-container-t6qd9 > <https://jenkins.ovirt.org/computer/loader-container-t6qd9/> (offline) > (suspended) > loader-container-vb5dj > <https://jenkins.ovirt.org/computer/loader-container-vb5dj/> (offline) > (suspended) > loader-container-vhdck > <https://jenkins.ovirt.org/computer/loader-container-vhdck/> (offline) > (suspended) > loader-container-vmww3 > <https://jenkins.ovirt.org/computer/loader-container-vmww3/> (offline) > (suspended) > loader-container-vv202 > <https://jenkins.ovirt.org/computer/loader-container-vv202/> (offline) > (suspended) > loader-container-w7n89 > <https://jenkins.ovirt.org/computer/loader-container-w7n89/> (offline) > (suspended) > loader-container-x5gwl > <https://jenkins.ovirt.org/computer/loader-container-x5gwl/> (offline) > (suspended) > loader-container-xmthb > <https://jenkins.ovirt.org/computer/loader-container-xmthb/> (offline) > (suspended) > loader-container-xxcl1 > <https://jenkins.ovirt.org/computer/loader-container-xxcl1/> (offline) > (suspended) > loader-container-zp4w9 > <https://jenkins.ovirt.org/computer/loader-container-zp4w9/> (offline) > (suspended) > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7YBUQRA6HSAG4… > -- Eyal edri He / Him / His MANAGER CONTINUOUS PRODUCTIZATION SYSTEM ENGINEERING Red Hat <https://www.redhat.com/> <https://red.ht/sig> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel) > All loader-container nodes are offline, 99 jobs in the queue, jobs stuck for 2 hours > ------------------------------------------------------------------------------------ > > Key: OVIRT-2768 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2768 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > We depend on jenkins to run our tests and build artifacts in a timely > manner. > Here is example job, stuck for 2 hours: > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/9212/ > This looks like the issue: > [image: collapse] > <https://jenkins.ovirt.org/toggleCollapse?paneId=executors>Build Executor > Status <https://jenkins.ovirt.org/computer/> > master <https://jenkins.ovirt.org/computer/(master)/> > 1 Idle > 2 Idle > 3 Idle > 4 Idle > lfedora1.lf-dev.marist.edu > <https://jenkins.ovirt.org/computer/lfedora1.lf-dev.marist.edu/> > 1 Idle > loader-container-28ljp > <https://jenkins.ovirt.org/computer/loader-container-28ljp/> (offline) > (suspended) > loader-container-35tlc > <https://jenkins.ovirt.org/computer/loader-container-35tlc/> (offline) > (suspended) > loader-container-3j0ps > <https://jenkins.ovirt.org/computer/loader-container-3j0ps/> (offline) > (suspended) > loader-container-4391q > <https://jenkins.ovirt.org/computer/loader-container-4391q/> (offline) > (suspended) > loader-container-58v3d > <https://jenkins.ovirt.org/computer/loader-container-58v3d/> (offline) > (suspended) > loader-container-66r5r > <https://jenkins.ovirt.org/computer/loader-container-66r5r/> (offline) > (suspended) > loader-container-6k57j > <https://jenkins.ovirt.org/computer/loader-container-6k57j/> (offline) > (suspended) > loader-container-7813f > <https://jenkins.ovirt.org/computer/loader-container-7813f/> (offline) > (suspended) > loader-container-8dkj1 > <https://jenkins.ovirt.org/computer/loader-container-8dkj1/> (offline) > (suspended) > loader-container-8hb98 > <https://jenkins.ovirt.org/computer/loader-container-8hb98/> (offline) > (suspended) > loader-container-8v609 > <https://jenkins.ovirt.org/computer/loader-container-8v609/> (offline) > (suspended) > loader-container-99f36 > <https://jenkins.ovirt.org/computer/loader-container-99f36/> (offline) > (suspended) > loader-container-c16rq > <https://jenkins.ovirt.org/computer/loader-container-c16rq/> (offline) > (suspended) > loader-container-c1vkj > <https://jenkins.ovirt.org/computer/loader-container-c1vkj/> (offline) > (suspended) > loader-container-c4g8c > <https://jenkins.ovirt.org/computer/loader-container-c4g8c/> (offline) > (suspended) > loader-container-cqrx5 > <https://jenkins.ovirt.org/computer/loader-container-cqrx5/> (offline) > (suspended) > loader-container-d1r8d > <https://jenkins.ovirt.org/computer/loader-container-d1r8d/> (offline) > (suspended) > loader-container-dpp70 > <https://jenkins.ovirt.org/computer/loader-container-dpp70/> (offline) > (suspended) > loader-container-gz52q > <https://jenkins.ovirt.org/computer/loader-container-gz52q/> (offline) > (suspended) > loader-container-h9kcf > <https://jenkins.ovirt.org/computer/loader-container-h9kcf/> (offline) > (suspended) > loader-container-hjq0g > <https://jenkins.ovirt.org/computer/loader-container-hjq0g/> (offline) > (suspended) > loader-container-ht75p > <https://jenkins.ovirt.org/computer/loader-container-ht75p/> (offline) > (suspended) > loader-container-htstk > <https://jenkins.ovirt.org/computer/loader-container-htstk/> (offline) > (suspended) > loader-container-j47s3 > <https://jenkins.ovirt.org/computer/loader-container-j47s3/> (offline) > (suspended) > loader-container-k2p10 > <https://jenkins.ovirt.org/computer/loader-container-k2p10/> (offline) > (suspended) > loader-container-kc066 > <https://jenkins.ovirt.org/computer/loader-container-kc066/> (offline) > (suspended) > loader-container-l1vd9 > <https://jenkins.ovirt.org/computer/loader-container-l1vd9/> (offline) > (suspended) > loader-container-l7lqn > <https://jenkins.ovirt.org/computer/loader-container-l7lqn/> (offline) > (suspended) > loader-container-lqqkj > <https://jenkins.ovirt.org/computer/loader-container-lqqkj/> (offline) > (suspended) > loader-container-m8fkq > <https://jenkins.ovirt.org/computer/loader-container-m8fkq/> (offline) > (suspended) > loader-container-n2zml > <https://jenkins.ovirt.org/computer/loader-container-n2zml/> (offline) > (suspended) > loader-container-p7rxw > <https://jenkins.ovirt.org/computer/loader-container-p7rxw/> (offline) > (suspended) > loader-container-pqg56 > <https://jenkins.ovirt.org/computer/loader-container-pqg56/> (offline) > (suspended) > loader-container-qtsdr > <https://jenkins.ovirt.org/computer/loader-container-qtsdr/> (offline) > (suspended) > loader-container-qxwnp > <https://jenkins.ovirt.org/computer/loader-container-qxwnp/> (offline) > (suspended) > loader-container-rdlww > <https://jenkins.ovirt.org/computer/loader-container-rdlww/> (offline) > (suspended) > loader-container-rp1rc > <https://jenkins.ovirt.org/computer/loader-container-rp1rc/> (offline) > (suspended) > loader-container-s6ndk > <https://jenkins.ovirt.org/computer/loader-container-s6ndk/> (offline) > (suspended) > loader-container-sv0pt > <https://jenkins.ovirt.org/computer/loader-container-sv0pt/> (offline) > (suspended) > loader-container-t6qd9 > <https://jenkins.ovirt.org/computer/loader-container-t6qd9/> (offline) > (suspended) > loader-container-vb5dj > <https://jenkins.ovirt.org/computer/loader-container-vb5dj/> (offline) > (suspended) > loader-container-vhdck > <https://jenkins.ovirt.org/computer/loader-container-vhdck/> (offline) > (suspended) > loader-container-vmww3 > <https://jenkins.ovirt.org/computer/loader-container-vmww3/> (offline) > (suspended) > loader-container-vv202 > <https://jenkins.ovirt.org/computer/loader-container-vv202/> (offline) > (suspended) > loader-container-w7n89 > <https://jenkins.ovirt.org/computer/loader-container-w7n89/> (offline) > (suspended) > loader-container-x5gwl > <https://jenkins.ovirt.org/computer/loader-container-x5gwl/> (offline) > (suspended) > loader-container-xmthb > <https://jenkins.ovirt.org/computer/loader-container-xmthb/> (offline) > (suspended) > loader-container-xxcl1 > <https://jenkins.ovirt.org/computer/loader-container-xxcl1/> (offline) > (suspended) > loader-container-zp4w9 > <https://jenkins.ovirt.org/computer/loader-container-zp4w9/> (offline) > (suspended) -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100106)

1 0

[CQ]: 102328,1 (ovirt-appliance) failed "ovirt-4.3" system tests
by oVirt Jenkins 01 Aug '19

01 Aug '19

Change 102328,1 (ovirt-appliance) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/102328/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/1639/

1 0

[CQ]: e68aa26 (ovirt-ansible-hosted-engine-setup) failed "ovirt-4.2" system tests
by oVirt Jenkins 01 Aug '19

01 Aug '19

Change e68aa26 (ovirt-ansible-hosted-engine-setup) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/e68aa26a2… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/4671/

1 0

[CQ]: e68aa26 (ovirt-ansible-hosted-engine-setup) failed "ovirt-master" system tests
by oVirt Jenkins 01 Aug '19

01 Aug '19

Change e68aa26 (ovirt-ansible-hosted-engine-setup) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/e68aa26a2… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15226/

1 0

Build failed in Jenkins: system-sync_mirrors-glusterfs-5-el7-x86_64 #580
by jenkins＠jenkins.phx.ovirt.org 01 Aug '19

01 Aug '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-glusterfs-5-el7-x86_64/580…> ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-glusterfs-5-el7-x86_64/ws/> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 00bee591a6c05d78885a471c2c4930c93fedaed1 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 00bee591a6c05d78885a471c2c4930c93fedaed1 Commit message: "Increase git clone timeout to 20 minutes" > git rev-list --no-walk 00bee591a6c05d78885a471c2c4930c93fedaed1 # timeout=10 [system-sync_mirrors-glusterfs-5-el7-x86_64] $ /bin/bash -xe /tmp/jenkins5171295817777623537.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror glusterfs-5-el7 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror glusterfs-5-el7 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror glusterfs-5-el7 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=glusterfs-5-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs glusterfs-5-el7 yum + local repo_name=glusterfs-5-el7 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/glusterfs-5-el7 /var/www/html/repos/yum/glusterfs-5-el7/base + check_yum_sync_needed glusterfs-5-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=glusterfs-5-el7 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/glusterfs-5-el7 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync glusterfs-5-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=glusterfs-5-el7 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=glusterfs-5-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/glusterfs-5-el7/base --norepopath --newest-only --urls --quiet Traceback (most recent call last): File "/usr/bin/reposync", line 373, in <module> main() File "/usr/bin/reposync", line 185, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1468, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 777, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 760, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 738, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 715, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-qemu-ev-release-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-qemu-ev-release-el7' + reposync_out= Build step 'Execute shell' marked build as failure

1 1

[JIRA] (OVIRT-2768) All loader-container nodes are offline, 99 jobs in the queue, jobs stuck for 2 hours
by Eyal Edri (oVirt JIRA) 01 Aug '19

01 Aug '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2768?page=com.atlassian.jira.… ] Eyal Edri commented on OVIRT-2768: ---------------------------------- [~accountid:557058:caa507e4-2696-4f45-8da5-d2585a4bb794] can you add info on what was the bug and how you fixed it? > All loader-container nodes are offline, 99 jobs in the queue, jobs stuck for 2 hours > ------------------------------------------------------------------------------------ > > Key: OVIRT-2768 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2768 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > We depend on jenkins to run our tests and build artifacts in a timely > manner. > Here is example job, stuck for 2 hours: > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/9212/ > This looks like the issue: > [image: collapse] > <https://jenkins.ovirt.org/toggleCollapse?paneId=executors>Build Executor > Status <https://jenkins.ovirt.org/computer/> > master <https://jenkins.ovirt.org/computer/(master)/> > 1 Idle > 2 Idle > 3 Idle > 4 Idle > lfedora1.lf-dev.marist.edu > <https://jenkins.ovirt.org/computer/lfedora1.lf-dev.marist.edu/> > 1 Idle > loader-container-28ljp > <https://jenkins.ovirt.org/computer/loader-container-28ljp/> (offline) > (suspended) > loader-container-35tlc > <https://jenkins.ovirt.org/computer/loader-container-35tlc/> (offline) > (suspended) > loader-container-3j0ps > <https://jenkins.ovirt.org/computer/loader-container-3j0ps/> (offline) > (suspended) > loader-container-4391q > <https://jenkins.ovirt.org/computer/loader-container-4391q/> (offline) > (suspended) > loader-container-58v3d > <https://jenkins.ovirt.org/computer/loader-container-58v3d/> (offline) > (suspended) > loader-container-66r5r > <https://jenkins.ovirt.org/computer/loader-container-66r5r/> (offline) > (suspended) > loader-container-6k57j > <https://jenkins.ovirt.org/computer/loader-container-6k57j/> (offline) > (suspended) > loader-container-7813f > <https://jenkins.ovirt.org/computer/loader-container-7813f/> (offline) > (suspended) > loader-container-8dkj1 > <https://jenkins.ovirt.org/computer/loader-container-8dkj1/> (offline) > (suspended) > loader-container-8hb98 > <https://jenkins.ovirt.org/computer/loader-container-8hb98/> (offline) > (suspended) > loader-container-8v609 > <https://jenkins.ovirt.org/computer/loader-container-8v609/> (offline) > (suspended) > loader-container-99f36 > <https://jenkins.ovirt.org/computer/loader-container-99f36/> (offline) > (suspended) > loader-container-c16rq > <https://jenkins.ovirt.org/computer/loader-container-c16rq/> (offline) > (suspended) > loader-container-c1vkj > <https://jenkins.ovirt.org/computer/loader-container-c1vkj/> (offline) > (suspended) > loader-container-c4g8c > <https://jenkins.ovirt.org/computer/loader-container-c4g8c/> (offline) > (suspended) > loader-container-cqrx5 > <https://jenkins.ovirt.org/computer/loader-container-cqrx5/> (offline) > (suspended) > loader-container-d1r8d > <https://jenkins.ovirt.org/computer/loader-container-d1r8d/> (offline) > (suspended) > loader-container-dpp70 > <https://jenkins.ovirt.org/computer/loader-container-dpp70/> (offline) > (suspended) > loader-container-gz52q > <https://jenkins.ovirt.org/computer/loader-container-gz52q/> (offline) > (suspended) > loader-container-h9kcf > <https://jenkins.ovirt.org/computer/loader-container-h9kcf/> (offline) > (suspended) > loader-container-hjq0g > <https://jenkins.ovirt.org/computer/loader-container-hjq0g/> (offline) > (suspended) > loader-container-ht75p > <https://jenkins.ovirt.org/computer/loader-container-ht75p/> (offline) > (suspended) > loader-container-htstk > <https://jenkins.ovirt.org/computer/loader-container-htstk/> (offline) > (suspended) > loader-container-j47s3 > <https://jenkins.ovirt.org/computer/loader-container-j47s3/> (offline) > (suspended) > loader-container-k2p10 > <https://jenkins.ovirt.org/computer/loader-container-k2p10/> (offline) > (suspended) > loader-container-kc066 > <https://jenkins.ovirt.org/computer/loader-container-kc066/> (offline) > (suspended) > loader-container-l1vd9 > <https://jenkins.ovirt.org/computer/loader-container-l1vd9/> (offline) > (suspended) > loader-container-l7lqn > <https://jenkins.ovirt.org/computer/loader-container-l7lqn/> (offline) > (suspended) > loader-container-lqqkj > <https://jenkins.ovirt.org/computer/loader-container-lqqkj/> (offline) > (suspended) > loader-container-m8fkq > <https://jenkins.ovirt.org/computer/loader-container-m8fkq/> (offline) > (suspended) > loader-container-n2zml > <https://jenkins.ovirt.org/computer/loader-container-n2zml/> (offline) > (suspended) > loader-container-p7rxw > <https://jenkins.ovirt.org/computer/loader-container-p7rxw/> (offline) > (suspended) > loader-container-pqg56 > <https://jenkins.ovirt.org/computer/loader-container-pqg56/> (offline) > (suspended) > loader-container-qtsdr > <https://jenkins.ovirt.org/computer/loader-container-qtsdr/> (offline) > (suspended) > loader-container-qxwnp > <https://jenkins.ovirt.org/computer/loader-container-qxwnp/> (offline) > (suspended) > loader-container-rdlww > <https://jenkins.ovirt.org/computer/loader-container-rdlww/> (offline) > (suspended) > loader-container-rp1rc > <https://jenkins.ovirt.org/computer/loader-container-rp1rc/> (offline) > (suspended) > loader-container-s6ndk > <https://jenkins.ovirt.org/computer/loader-container-s6ndk/> (offline) > (suspended) > loader-container-sv0pt > <https://jenkins.ovirt.org/computer/loader-container-sv0pt/> (offline) > (suspended) > loader-container-t6qd9 > <https://jenkins.ovirt.org/computer/loader-container-t6qd9/> (offline) > (suspended) > loader-container-vb5dj > <https://jenkins.ovirt.org/computer/loader-container-vb5dj/> (offline) > (suspended) > loader-container-vhdck > <https://jenkins.ovirt.org/computer/loader-container-vhdck/> (offline) > (suspended) > loader-container-vmww3 > <https://jenkins.ovirt.org/computer/loader-container-vmww3/> (offline) > (suspended) > loader-container-vv202 > <https://jenkins.ovirt.org/computer/loader-container-vv202/> (offline) > (suspended) > loader-container-w7n89 > <https://jenkins.ovirt.org/computer/loader-container-w7n89/> (offline) > (suspended) > loader-container-x5gwl > <https://jenkins.ovirt.org/computer/loader-container-x5gwl/> (offline) > (suspended) > loader-container-xmthb > <https://jenkins.ovirt.org/computer/loader-container-xmthb/> (offline) > (suspended) > loader-container-xxcl1 > <https://jenkins.ovirt.org/computer/loader-container-xxcl1/> (offline) > (suspended) > loader-container-zp4w9 > <https://jenkins.ovirt.org/computer/loader-container-zp4w9/> (offline) > (suspended) -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100106)

1 0