Build failed in Jenkins:
system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64 #79
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7...>
Changes:
[Daniel Belenky] usrc.py: allow manual updates of upstream sources
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-sclo-rh-release-7...>
No credentials specified
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Pruning obsolete local branches
Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git
> git --version # timeout=10
> git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 9c6683c529b31a0f471a5690de84fe7bf590a5d8 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9c6683c529b31a0f471a5690de84fe7bf590a5d8
Commit message: "usrc.py: allow manual updates of upstream sources"
> git rev-list --no-walk 91e005fd78b159d32a217316013437f760918664 # timeout=10
[system-sync_mirrors-centos-sclo-rh-release-7.6-el7-x86_64] $ /bin/bash -xe /tmp/jenkins7697791391914460797.sh
+ jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ MIRRORS_MP_BASE=/var/www/html/repos
+ MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos
+ MIRRORS_CACHE=/home/jenkins/mirrors_cache
+ MAX_LOCK_ATTEMPTS=120
+ LOCK_WAIT_INTERVAL=5
+ LOCK_BASE=/home/jenkins
+ OLD_MD_TO_KEEP=100
+ HTTP_SELINUX_TYPE=httpd_sys_content_t
+ HTTP_FILE_MODE=644
+ main resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ local command=resync_yum_mirror
+ command_args=("${@:2}")
+ local command_args
+ cmd_resync_yum_mirror centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ local repo_name=centos-sclo-rh-release-7.6-el7
+ local repo_archs=x86_64
+ local reposync_conf=jenkins/data/mirrors-reposync.conf
+ local sync_needed
+ mkdir -p /home/jenkins/mirrors_cache
+ verify_repo_fs centos-sclo-rh-release-7.6-el7 yum
+ local repo_name=centos-sclo-rh-release-7.6-el7
+ local repo_type=yum
+ sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7 /var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base
+ check_yum_sync_needed centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed
+ local repo_name=centos-sclo-rh-release-7.6-el7
+ local repo_archs=x86_64
+ local reposync_conf=jenkins/data/mirrors-reposync.conf
+ local p_sync_needed=sync_needed
+ local reposync_out
+ echo 'Checking if mirror needs a resync'
Checking if mirror needs a resync
+ rm -rf /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7
++ IFS=,
++ echo x86_64
+ for arch in '$(IFS=,; echo $repo_archs)'
++ run_reposync centos-sclo-rh-release-7.6-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet
++ local repo_name=centos-sclo-rh-release-7.6-el7
++ local repo_arch=x86_64
++ local reposync_conf=jenkins/data/mirrors-reposync.conf
++ extra_args=("${@:4}")
++ local extra_args
++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-sclo-rh-release-7.6-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-sclo-rh-release-7.6-el7/base --norepopath --newest-only --urls --quiet
Error setting up repositories: Error making cache directory: /home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-sclo-rh-release-7.6-el7'
+ reposync_out=
Build step 'Execute shell' marked build as failure
5 years
[JIRA] (OVIRT-2837) OST fails for collecting artifacts
by Marcin Sobczyk (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2837?page=com.atlassian.jir... ]
Marcin Sobczyk commented on OVIRT-2837:
---------------------------------------
Posted a patch for this issue: [https://gerrit.ovirt.org/#/c/104789/|https://gerrit.ovirt.org/#/c/104789/]
> OST fails for collecting artifacts
> ----------------------------------
>
> Key: OVIRT-2837
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2837
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: Amit Bawer
> Assignee: infra
>
> Happens for several runs, full log can be seen at
> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/expo...
> 2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR::
> - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in
> 0:00:08)
> 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG::
> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in
> _collect_artifacts
> vm.collect_artifacts(path, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 748, in collect_artifacts
> ignore_nopath=ignore_nopath
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 468, in extract_paths
> return self.provider.extract_paths(paths, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py",
> line 398, in extract_paths
> ignore_nopath=ignore_nopath,
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 253, in extract_paths
> self._extract_paths_tar_gz(paths, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 102, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 341, in _extract_paths_tar_gz
> raise ExtractPathNoPathError(remote_path)
> 2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error
> while running thread Thread-42
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
> _ret_via_queue
> queue.put({'return': func()})
> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526,
> in _collect_artifacts
> vm.collect_artifacts(path, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 748, in collect_artifacts
> ignore_nopath=ignore_nopath
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 468, in extract_paths
> return self.provider.extract_paths(paths, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py",
> line 398, in extract_paths
> ignore_nopath=ignore_nopath,
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 253, in extract_paths
> self._extract_paths_tar_gz(paths, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 102, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 341, in _extract_paths_tar_gz
> raise ExtractPathNoPathError(remote_path)
> ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
5 years
[JIRA] (OVIRT-2837) OST fails for collecting artifacts
by Marcin Sobczyk (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2837?page=com.atlassian.jir... ]
Marcin Sobczyk commented on OVIRT-2837:
---------------------------------------
Hi,
I thought it's been removed already, but it seems it's not.
I'm the author of the currently used and newer implementation of
artifact collection in lago,
but from my experiments, I've learned that the extraction of wildcard
paths never worked.
Here's an email I wrote to Galit about it some time ago:
=================================
Hi Galit,
aaah yes - wildcard collection doesn't work - it never worked, even
before my changes.
TL; DR - we just need to remove wildcard stuff from "LagoInitFile.in"
files ("/tmp/otopi*", "/tmp/ovirt*").
If you're curious what really happens... :)
The old algorithm uses "SCPClient" from "scp" library to copy files.
"scp.get" function accepts, among others, two arguments - "remote_path"
and "local_path".
What we do, is we change slashes in "remote_path" to underscores and
pass it as "local_path":
https://github.com/lago-project/lago/blob/7024fb7cabbd4ebc87f3ad12e35e0f5...
The effect is, we command "scp.get" to retrieve "/tmp/otopi*" and save
it as "_tmp_otopi*"... which of course makes no sense at all and doesn't
work...
The new implementation *could* work with wildcards because the
collection is divided into two stages:
https://github.com/lago-project/lago/blob/9803eeacd41b3f91cd6661a110aa028...
First we do the "tar -> copy tar with ssh -> untar to tmpdir" thing and
*only then* we use "shutil.move" to rename the files to the underscored
version.
We could use "glob" module to try to iterate over stuff like
"/tmp/otopi*" and rename the files appropriately.
However, we maintain two parallel implementations of artifacts
collection - the old one being a "plan B" in case there's no "tar" or
"gzip" on the target machine.
This is the reason we have to keep both implementations identical in
behavior to avoid confusion. BTW the new implementation could drop the
underscore-renaming process completely - I think the only reason we do
the renaming in the old algorithm is because "scp" won't create
intermediary directories for you... untarring stuff handles that case
well, but that's a backwards-compatibility-breaking change :)
=================================
I will post a patch that removes this.
Regards, Marcin
On 11/18/19 2:45 PM, Amit Bawer wrote:
> Happens for several runs, full log can be seen at
> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/expo...
> 2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08)
> 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts
> vm.collect_artifacts(path, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts
> ignore_nopath=ignore_nopath
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths
> return self.provider.extract_paths(paths, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths
> ignore_nopath=ignore_nopath,
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths
> self._extract_paths_tar_gz(paths, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz
> raise ExtractPathNoPathError(remote_path)
>
> 2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while running thread Thread-42
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue
> queue.put({'return': func()})
> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts
> vm.collect_artifacts(path, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts
> ignore_nopath=ignore_nopath
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths
> return self.provider.extract_paths(paths, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths
> ignore_nopath=ignore_nopath,
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths
> self._extract_paths_tar_gz(paths, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz
> raise ExtractPathNoPathError(remote_path)
> ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
>
> _______________________________________________
> Devel mailing list -- devel(a)ovirt.org
> To unsubscribe send an email to devel-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
> List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PCS27XT3ODF...
> OST fails for collecting artifacts
> ----------------------------------
>
> Key: OVIRT-2837
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2837
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: Amit Bawer
> Assignee: infra
>
> Happens for several runs, full log can be seen at
> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/expo...
> 2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR::
> - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in
> 0:00:08)
> 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG::
> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in
> _collect_artifacts
> vm.collect_artifacts(path, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 748, in collect_artifacts
> ignore_nopath=ignore_nopath
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 468, in extract_paths
> return self.provider.extract_paths(paths, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py",
> line 398, in extract_paths
> ignore_nopath=ignore_nopath,
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 253, in extract_paths
> self._extract_paths_tar_gz(paths, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 102, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 341, in _extract_paths_tar_gz
> raise ExtractPathNoPathError(remote_path)
> 2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error
> while running thread Thread-42
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
> _ret_via_queue
> queue.put({'return': func()})
> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526,
> in _collect_artifacts
> vm.collect_artifacts(path, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 748, in collect_artifacts
> ignore_nopath=ignore_nopath
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 468, in extract_paths
> return self.provider.extract_paths(paths, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py",
> line 398, in extract_paths
> ignore_nopath=ignore_nopath,
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 253, in extract_paths
> self._extract_paths_tar_gz(paths, ignore_nopath)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 102, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
> 341, in _extract_paths_tar_gz
> raise ExtractPathNoPathError(remote_path)
> ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
5 years
