[JIRA] (OVIRT-2814) reposync fails syncing repos on completely
unrelated paths (concurrency issue?)
by Sandro Bonazzola (oVirt JIRA)
Sandro Bonazzola created OVIRT-2814:
---------------------------------------
Summary: reposync fails syncing repos on completely unrelated paths (concurrency issue?)
Key: OVIRT-2814
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: Sandro Bonazzola
Assignee: infra
About job
https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x...
As you can see below
05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
--repoid=centos-kvm-common-el7 --arch=x86_64
--cachedir=/home/jenkins/mirrors_cache
--download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
--norepopath --newest-only --urls --quiet
the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
using as cache directory "/home/jenkins/mirrors_cache"
but in "/home/jenkins/mirrors_cache" there's
"/home/jenkins/mirrors_cache/fedora-base-fc29":
05:35:16 Traceback (most recent call last):
05:35:16 File "/usr/bin/reposync", line 373, in <module>
05:35:16 main()
05:35:16 File "/usr/bin/reposync", line 185, in main
05:35:16 my.doRepoSetup()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
681, in doRepoSetup
05:35:16 return self._getRepos(thisrepo, True)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
721, in _getRepos
05:35:16 self._repos.doSetup(thisrepo)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
in doSetup
05:35:16 self.retrieveAllMD()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
in retrieveAllMD
05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
1468, in _commonLoadRepoXML
05:35:16 local = self.cachedir + '/repomd.xml'
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
777, in <lambda>
05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
760, in _dirGetAttr
05:35:16 self.dirSetup()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
738, in dirSetup
05:35:16 self._dirSetupMkdir_p(dir)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
715, in _dirSetupMkdir_p
05:35:16 raise Errors.RepoError, msg
05:35:16 yum.Errors.RepoError: Error making cache directory:
/home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
this looks like a concurrency issue, a lock should be used in order to
prevent two instances to use the same cache directory at the same time or
use separate cache directories for different repos.
--
Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100113)
5 years, 1 month
Build failed in Jenkins:
system-sync_mirrors-centos-kvm-common-el7-x86_64 #2934
by jenkins@jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x8...>
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x8...>
No credentials specified
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Pruning obsolete local branches
Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git
> git --version # timeout=10
> git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a796094bfd15f1a85b9f630aaeb25ce6e0dab35d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a796094bfd15f1a85b9f630aaeb25ce6e0dab35d
Commit message: "node-ng-image: drop fc30 jobs on 4.3"
> git rev-list --no-walk a796094bfd15f1a85b9f630aaeb25ce6e0dab35d # timeout=10
[system-sync_mirrors-centos-kvm-common-el7-x86_64] $ /bin/bash -xe /tmp/jenkins3448724163555223690.sh
+ jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-kvm-common-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ MIRRORS_MP_BASE=/var/www/html/repos
+ MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos
+ MIRRORS_CACHE=/home/jenkins/mirrors_cache
+ MAX_LOCK_ATTEMPTS=120
+ LOCK_WAIT_INTERVAL=5
+ LOCK_BASE=/home/jenkins
+ OLD_MD_TO_KEEP=100
+ HTTP_SELINUX_TYPE=httpd_sys_content_t
+ HTTP_FILE_MODE=644
+ main resync_yum_mirror centos-kvm-common-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ local command=resync_yum_mirror
+ command_args=("${@:2}")
+ local command_args
+ cmd_resync_yum_mirror centos-kvm-common-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ local repo_name=centos-kvm-common-el7
+ local repo_archs=x86_64
+ local reposync_conf=jenkins/data/mirrors-reposync.conf
+ local sync_needed
+ mkdir -p /home/jenkins/mirrors_cache
+ verify_repo_fs centos-kvm-common-el7 yum
+ local repo_name=centos-kvm-common-el7
+ local repo_type=yum
+ sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-kvm-common-el7 /var/www/html/repos/yum/centos-kvm-common-el7/base
+ check_yum_sync_needed centos-kvm-common-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed
+ local repo_name=centos-kvm-common-el7
+ local repo_archs=x86_64
+ local reposync_conf=jenkins/data/mirrors-reposync.conf
+ local p_sync_needed=sync_needed
+ local reposync_out
+ echo 'Checking if mirror needs a resync'
Checking if mirror needs a resync
+ rm -rf /home/jenkins/mirrors_cache/centos-kvm-common-el7
++ IFS=,
++ echo x86_64
+ for arch in '$(IFS=,; echo $repo_archs)'
++ run_reposync centos-kvm-common-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet
++ local repo_name=centos-kvm-common-el7
++ local repo_arch=x86_64
++ local reposync_conf=jenkins/data/mirrors-reposync.conf
++ extra_args=("${@:4}")
++ local extra_args
++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-kvm-common-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base --norepopath --newest-only --urls --quiet
Traceback (most recent call last):
File "/usr/bin/reposync", line 373, in <module>
main()
File "/usr/bin/reposync", line 185, in main
my.doRepoSetup()
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup
return self._getRepos(thisrepo, True)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos
self._repos.doSetup(thisrepo)
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup
self.retrieveAllMD()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD
dl = repo._async and repo._commonLoadRepoXML(repo)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1468, in _commonLoadRepoXML
local = self.cachedir + '/repomd.xml'
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 777, in <lambda>
cachedir = property(lambda self: self._dirGetAttr('cachedir'))
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 760, in _dirGetAttr
self.dirSetup()
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 738, in dirSetup
self._dirSetupMkdir_p(dir)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 715, in _dirSetupMkdir_p
raise Errors.RepoError, msg
yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
+ reposync_out=
Build step 'Execute shell' marked build as failure
5 years, 1 month