8:38 a.m.
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2837?page=com.atlassian.jir...
]
Marcin Sobczyk commented on OVIRT-2837:
---------------------------------------
Hi,
I thought it's been removed already, but it seems it's not.
I'm the author of the currently used and newer implementation of
artifact collection in lago,
but from my experiments, I've learned that the extraction of wildcard
paths never worked.
Here's an email I wrote to Galit about it some time ago:
=================================
Hi Galit,
aaah yes - wildcard collection doesn't work - it never worked, even
before my changes.
TL; DR - we just need to remove wildcard stuff from "LagoInitFile.in"
files ("/tmp/otopi*", "/tmp/ovirt*").
If you're curious what really happens... :)
The old algorithm uses "SCPClient" from "scp" library to copy files.
"scp.get" function accepts, among others, two arguments -
"remote_path"
and "local_path".
What we do, is we change slashes in "remote_path" to underscores and
pass it as "local_path":
https://github.com/lago-project/lago/blob/7024fb7cabbd4ebc87f3ad12e35e0f5...
The effect is, we command "scp.get" to retrieve "/tmp/otopi*" and save
it as "_tmp_otopi*"... which of course makes no sense at all and doesn't
work...
The new implementation *could* work with wildcards because the
collection is divided into two stages:
https://github.com/lago-project/lago/blob/9803eeacd41b3f91cd6661a110aa028...
First we do the "tar -> copy tar with ssh -> untar to tmpdir" thing and
*only then* we use "shutil.move" to rename the files to the underscored
version.
We could use "glob" module to try to iterate over stuff like
"/tmp/otopi*" and rename the files appropriately.
However, we maintain two parallel implementations of artifacts
collection - the old one being a "plan B" in case there's no "tar"
or
"gzip" on the target machine.
This is the reason we have to keep both implementations identical in
behavior to avoid confusion. BTW the new implementation could drop the
underscore-renaming process completely - I think the only reason we do
the renaming in the old algorithm is because "scp" won't create
intermediary directories for you... untarring stuff handles that case
well, but that's a backwards-compatibility-breaking change :)
=================================
I will post a patch that removes this.
Regards, Marcin
On 11/18/19 2:45 PM, Amit Bawer wrote:
Happens for several runs, full log can be seen at
http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/expo...
2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: -
[Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08)
2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File
"/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in
_collect_artifacts
vm.collect_artifacts(path, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in
collect_artifacts
ignore_nopath=ignore_nopath
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in
extract_paths
return self.provider.extract_paths(paths, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line
398, in extract_paths
ignore_nopath=ignore_nopath,
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in
extract_paths
self._extract_paths_tar_gz(paths, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in
wrapper
return func(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in
_extract_paths_tar_gz
raise ExtractPathNoPathError(remote_path)
2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while
running thread Thread-42
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
_ret_via_queue
queue.put({'return': func()})
File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in
_collect_artifacts
vm.collect_artifacts(path, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in
collect_artifacts
ignore_nopath=ignore_nopath
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in
extract_paths
return self.provider.extract_paths(paths, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line
398, in extract_paths
ignore_nopath=ignore_nopath,
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in
extract_paths
self._extract_paths_tar_gz(paths, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in
wrapper
return func(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in
_extract_paths_tar_gz
raise ExtractPathNoPathError(remote_path)
ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
_______________________________________________
Devel mailing list -- devel(a)ovirt.org
To unsubscribe send an email to devel-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PCS27XT3ODF...
OST fails for collecting artifacts
----------------------------------
Key: OVIRT-2837
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2837
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: Amit Bawer
Assignee: infra
Happens for several runs, full log can be seen at
http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/expo...
2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR::
- [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in
0:00:08)
2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG::
File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in
_collect_artifacts
vm.collect_artifacts(path, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
748, in collect_artifacts
ignore_nopath=ignore_nopath
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
468, in extract_paths
return self.provider.extract_paths(paths, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py",
line 398, in extract_paths
ignore_nopath=ignore_nopath,
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
253, in extract_paths
self._extract_paths_tar_gz(paths, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
102, in wrapper
return func(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
341, in _extract_paths_tar_gz
raise ExtractPathNoPathError(remote_path)
2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error
while running thread Thread-42
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in
_ret_via_queue
queue.put({'return': func()})
File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526,
in _collect_artifacts
vm.collect_artifacts(path, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
748, in collect_artifacts
ignore_nopath=ignore_nopath
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
468, in extract_paths
return self.provider.extract_paths(paths, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py",
line 398, in extract_paths
ignore_nopath=ignore_nopath,
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
253, in extract_paths
self._extract_paths_tar_gz(paths, ignore_nopath)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
102, in wrapper
return func(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
341, in _extract_paths_tar_gz
raise ExtractPathNoPathError(remote_path)
ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
1834
days inactive
1834
days old
0 comments
1 participants
participants (1)
-
Marcin Sobczyk (oVirt JIRA)