OST fails for collecting artifacts
Happens for several runs, full log can be seen at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/exporte... 2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08) 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path) 2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while running thread Thread-42 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path) ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
Hi, I thought it's been removed already, but it seems it's not. I'm the author of the currently used and newer implementation of artifact collection in lago, but from my experiments, I've learned that the extraction of wildcard paths never worked. Here's an email I wrote to Galit about it some time ago: ================================= Hi Galit, aaah yes - wildcard collection doesn't work - it never worked, even before my changes. TL; DR - we just need to remove wildcard stuff from "LagoInitFile.in" files ("/tmp/otopi*", "/tmp/ovirt*"). If you're curious what really happens... :) The old algorithm uses "SCPClient" from "scp" library to copy files. "scp.get" function accepts, among others, two arguments - "remote_path" and "local_path". What we do, is we change slashes in "remote_path" to underscores and pass it as "local_path": https://github.com/lago-project/lago/blob/7024fb7cabbd4ebc87f3ad12e35e0f5832... The effect is, we command "scp.get" to retrieve "/tmp/otopi*" and save it as "_tmp_otopi*"... which of course makes no sense at all and doesn't work... The new implementation *could* work with wildcards because the collection is divided into two stages: https://github.com/lago-project/lago/blob/9803eeacd41b3f91cd6661a110aa0285aa... First we do the "tar -> copy tar with ssh -> untar to tmpdir" thing and *only then* we use "shutil.move" to rename the files to the underscored version. We could use "glob" module to try to iterate over stuff like "/tmp/otopi*" and rename the files appropriately. However, we maintain two parallel implementations of artifacts collection - the old one being a "plan B" in case there's no "tar" or "gzip" on the target machine. This is the reason we have to keep both implementations identical in behavior to avoid confusion. BTW the new implementation could drop the underscore-renaming process completely - I think the only reason we do the renaming in the old algorithm is because "scp" won't create intermediary directories for you... untarring stuff handles that case well, but that's a backwards-compatibility-breaking change :) ================================= I will post a patch that removes this. Regards, Marcin On 11/18/19 2:45 PM, Amit Bawer wrote:
Happens for several runs, full log can be seen at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/exporte... 2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08) 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path)
2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while running thread Thread-42 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path) ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PCS27XT3ODFL5Z...
On Mon, Nov 18, 2019 at 4:38 PM Marcin Sobczyk <msobczyk@redhat.com> wrote:
Hi,
I thought it's been removed already, but it seems it's not. I'm the author of the currently used and newer implementation of artifact collection in lago, but from my experiments, I've learned that the extraction of wildcard paths never worked. Here's an email I wrote to Galit about it some time ago:
=================================
Hi Galit,
aaah yes - wildcard collection doesn't work - it never worked, even before my changes.
TL; DR - we just need to remove wildcard stuff from "LagoInitFile.in" files ("/tmp/otopi*", "/tmp/ovirt*").
I'd like to clarify why these were added. When host-deploy runs, during "Add host", it writes its log file locally. It defaults to /tmp/otopi*.log, and we do not change that, so it uses the default. If/when it finishes successfully, the engine copies this log to /var/log/ovirt-engine/host-deploy on the engine machine, and host-deploy removes its log from /tmp. If it fails in the middle, it will (likely) leave its log in /tmp. We had failures in the past that were hard to diagnose, and therefore added /tmp/otopi*, to help debug future similar cases. We can instead configure otopi/host-deploy on the host to keep its logs by default elsewhere. E.g. by doing this on the _host_ before adding it: mkdir -p /var/log/otopi mkdir -p /etc/otopi.env.d echo 'export OTOPI_LOGDIR=/var/log/otopi' > /etc/otopi.env.d/99-logdir.env and then collect /var/log/otopi. If this solves the problem, fine with me to do this and remove /tmp/otopi*. That said, I think collecting wildcards can be useful, so perhaps we should fix this anyway. Also, /tmp/otopi* should normally do not exist, so it should almost always fail to collect that. Is this really a problem? Or just annoying to have to skip over when reading lago/ost logs?
If you're curious what really happens... :)
The old algorithm uses "SCPClient" from "scp" library to copy files. "scp.get" function accepts, among others, two arguments - "remote_path" and "local_path". What we do, is we change slashes in "remote_path" to underscores and pass it as "local_path":
https://github.com/lago-project/lago/blob/7024fb7cabbd4ebc87f3ad12e35e0f5832...
The effect is, we command "scp.get" to retrieve "/tmp/otopi*" and save it as "_tmp_otopi*"... which of course makes no sense at all and doesn't work...
The new implementation *could* work with wildcards because the collection is divided into two stages:
https://github.com/lago-project/lago/blob/9803eeacd41b3f91cd6661a110aa0285aa...
First we do the "tar -> copy tar with ssh -> untar to tmpdir" thing and *only then* we use "shutil.move" to rename the files to the underscored version. We could use "glob" module to try to iterate over stuff like "/tmp/otopi*" and rename the files appropriately. However, we maintain two parallel implementations of artifacts collection - the old one being a "plan B" in case there's no "tar" or "gzip" on the target machine. This is the reason we have to keep both implementations identical in behavior to avoid confusion. BTW the new implementation could drop the underscore-renaming process completely - I think the only reason we do the renaming in the old algorithm is because "scp" won't create intermediary directories for you... untarring stuff handles that case well, but that's a backwards-compatibility-breaking change :)
=================================
I will post a patch that removes this.
Regards, Marcin
On 11/18/19 2:45 PM, Amit Bawer wrote:
Happens for several runs, full log can be seen at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/exporte...
2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08) 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path)
2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while running thread Thread-42 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path) ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PCS27XT3ODFL5Z...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5BSU44MUSBKIKB...
-- Didi
Yedidyah Bar David <didi@redhat.com> writes:
On Mon, Nov 18, 2019 at 4:38 PM Marcin Sobczyk <msobczyk@redhat.com> wrote:
Hi,
I thought it's been removed already, but it seems it's not. I'm the author of the currently used and newer implementation of artifact collection in lago, but from my experiments, I've learned that the extraction of wildcard paths never worked. Here's an email I wrote to Galit about it some time ago:
=================================
Hi Galit,
aaah yes - wildcard collection doesn't work - it never worked, even before my changes.
TL; DR - we just need to remove wildcard stuff from "LagoInitFile.in" files ("/tmp/otopi*", "/tmp/ovirt*").
I'd like to clarify why these were added.
When host-deploy runs, during "Add host", it writes its log file locally. It defaults to /tmp/otopi*.log, and we do not change that, so it uses the default.
If/when it finishes successfully, the engine copies this log to /var/log/ovirt-engine/host-deploy on the engine machine, and host-deploy removes its log from /tmp.
If it fails in the middle, it will (likely) leave its log in /tmp.
We had failures in the past that were hard to diagnose, and therefore added /tmp/otopi*, to help debug future similar cases.
We can instead configure otopi/host-deploy on the host to keep its logs by default elsewhere. E.g. by doing this on the _host_ before adding it:
mkdir -p /var/log/otopi mkdir -p /etc/otopi.env.d echo 'export OTOPI_LOGDIR=/var/log/otopi' > /etc/otopi.env.d/99-logdir.env
and then collect /var/log/otopi.
If this solves the problem, fine with me to do this and remove /tmp/otopi*.
That said, I think collecting wildcards can be useful, so perhaps we should fix this anyway.
Also, /tmp/otopi* should normally do not exist, so it should almost always fail to collect that. Is this really a problem? Or just annoying to have to skip over when reading lago/ost logs?
It is at least confusing and annoying to see these kind of errors in OST logs. If it is expected to fail in some cases, it shouldn't be reported as error and should be silently ignored when the file doesn't exist. Thanks, Milan
If you're curious what really happens... :)
The old algorithm uses "SCPClient" from "scp" library to copy files. "scp.get" function accepts, among others, two arguments - "remote_path" and "local_path". What we do, is we change slashes in "remote_path" to underscores and pass it as "local_path":
https://github.com/lago-project/lago/blob/7024fb7cabbd4ebc87f3ad12e35e0f5832...
The effect is, we command "scp.get" to retrieve "/tmp/otopi*" and save it as "_tmp_otopi*"... which of course makes no sense at all and doesn't work...
The new implementation *could* work with wildcards because the collection is divided into two stages:
https://github.com/lago-project/lago/blob/9803eeacd41b3f91cd6661a110aa0285aa...
First we do the "tar -> copy tar with ssh -> untar to tmpdir" thing and *only then* we use "shutil.move" to rename the files to the underscored version. We could use "glob" module to try to iterate over stuff like "/tmp/otopi*" and rename the files appropriately. However, we maintain two parallel implementations of artifacts collection - the old one being a "plan B" in case there's no "tar" or "gzip" on the target machine. This is the reason we have to keep both implementations identical in behavior to avoid confusion. BTW the new implementation could drop the underscore-renaming process completely - I think the only reason we do the renaming in the old algorithm is because "scp" won't create intermediary directories for you... untarring stuff handles that case well, but that's a backwards-compatibility-breaking change :)
=================================
I will post a patch that removes this.
Regards, Marcin
On 11/18/19 2:45 PM, Amit Bawer wrote:
Happens for several runs, full log can be seen at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/exporte...
2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08) 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path)
2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while running thread Thread-42 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path) ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PCS27XT3ODFL5Z...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5BSU44MUSBKIKB...
On Wed, Nov 20, 2019 at 11:09 AM Milan Zamazal <mzamazal@redhat.com> wrote:
Yedidyah Bar David <didi@redhat.com> writes:
On Mon, Nov 18, 2019 at 4:38 PM Marcin Sobczyk <msobczyk@redhat.com> wrote:
Hi,
I thought it's been removed already, but it seems it's not. I'm the author of the currently used and newer implementation of artifact collection in lago, but from my experiments, I've learned that the extraction of wildcard paths never worked. Here's an email I wrote to Galit about it some time ago:
=================================
Hi Galit,
aaah yes - wildcard collection doesn't work - it never worked, even before my changes.
TL; DR - we just need to remove wildcard stuff from "LagoInitFile.in" files ("/tmp/otopi*", "/tmp/ovirt*").
I'd like to clarify why these were added.
When host-deploy runs, during "Add host", it writes its log file locally. It defaults to /tmp/otopi*.log, and we do not change that, so it uses the default.
If/when it finishes successfully, the engine copies this log to /var/log/ovirt-engine/host-deploy on the engine machine, and host-deploy removes its log from /tmp.
If it fails in the middle, it will (likely) leave its log in /tmp.
We had failures in the past that were hard to diagnose, and therefore added /tmp/otopi*, to help debug future similar cases.
We can instead configure otopi/host-deploy on the host to keep its logs by default elsewhere. E.g. by doing this on the _host_ before adding it:
mkdir -p /var/log/otopi mkdir -p /etc/otopi.env.d echo 'export OTOPI_LOGDIR=/var/log/otopi' > /etc/otopi.env.d/99-logdir.env
and then collect /var/log/otopi.
If this solves the problem, fine with me to do this and remove /tmp/otopi*.
That said, I think collecting wildcards can be useful, so perhaps we should fix this anyway.
Also, /tmp/otopi* should normally do not exist, so it should almost always fail to collect that. Is this really a problem? Or just annoying to have to skip over when reading lago/ost logs?
It is at least confusing and annoying to see these kind of errors in OST logs. If it is expected to fail in some cases, it shouldn't be reported as error and should be silently ignored when the file doesn't exist.
I have no idea if it's possible to tell lago in the init file that some artifacts are mandatory and some are optional. If not, perhaps we want such functionality. Either way, a different question is what to do if/when collection fails. It's not clear to me that a job should fail if only collection failed. Even if we set some as mandatory and they fail. And another item is that it should be very clear from the logs if failed collection also failed the job... All are things to be done in lago, I guess. My patch only added a single line to ovirt-system-tests, I didn't spend time on relevant lago code. Anyway, now pushed https://gerrit.ovirt.org/104827 . If it passes well, we can ignore current thread issue for some more time... Sorry for the noise and best regards,
Thanks, Milan
If you're curious what really happens... :)
The old algorithm uses "SCPClient" from "scp" library to copy files. "scp.get" function accepts, among others, two arguments - "remote_path" and "local_path". What we do, is we change slashes in "remote_path" to underscores and pass it as "local_path":
https://github.com/lago-project/lago/blob/7024fb7cabbd4ebc87f3ad12e35e0f5832...
The effect is, we command "scp.get" to retrieve "/tmp/otopi*" and save it as "_tmp_otopi*"... which of course makes no sense at all and doesn't work...
The new implementation *could* work with wildcards because the collection is divided into two stages:
https://github.com/lago-project/lago/blob/9803eeacd41b3f91cd6661a110aa0285aa...
First we do the "tar -> copy tar with ssh -> untar to tmpdir" thing and *only then* we use "shutil.move" to rename the files to the underscored version. We could use "glob" module to try to iterate over stuff like "/tmp/otopi*" and rename the files appropriately. However, we maintain two parallel implementations of artifacts collection - the old one being a "plan B" in case there's no "tar" or "gzip" on the target machine. This is the reason we have to keep both implementations identical in behavior to avoid confusion. BTW the new implementation could drop the underscore-renaming process completely - I think the only reason we do the renaming in the old algorithm is because "scp" won't create intermediary directories for you... untarring stuff handles that case well, but that's a backwards-compatibility-breaking change :)
=================================
I will post a patch that removes this.
Regards, Marcin
On 11/18/19 2:45 PM, Amit Bawer wrote:
Happens for several runs, full log can be seen at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/6057/artifact/exporte...
2019-11-18 12:28:12,710::log_utils.py::end_log_task::670::root::ERROR:: - [Thread-42] lago-basic-suite-master-engine: [31mERROR [0m (in 0:00:08) 2019-11-18 12:28:12,731::log_utils.py::__exit__::607::lago.prefix::DEBUG:: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path)
2019-11-18 12:28:12,731::utils.py::_ret_via_queue::63::lago.utils::DEBUG::Error while running thread Thread-42 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 748, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 468, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 253, in extract_paths self._extract_paths_tar_gz(paths, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 102, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 341, in _extract_paths_tar_gz raise ExtractPathNoPathError(remote_path) ExtractPathNoPathError: Failed to extract files: /tmp/otopi*
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PCS27XT3ODFL5Z...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5BSU44MUSBKIKB...
-- Didi
participants (4)
-
Amit Bawer -
Marcin Sobczyk -
Milan Zamazal -
Yedidyah Bar David