Hi all, bumping this thread due to an almost identical failure[1]: ovirt-log-collector/ovirt-log-collector-20161030053238.log:2016-10-30 05:33:09::ERROR::__main__::791::root:: Failed to collect logs from: 192.168.200.4; /bin/ls: /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.job.0: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.recover.1: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.task: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.recover.0: No such file or directory To ensure I've checked lago/OST, and couldn't find any stage where there is a reference to '/rhv' nor any manipulation to ovirt-log-collector, only customizations made is a 'ovirt-log-collector.conf' with user/password. The code that pulls the logs in OST[2] runs the following command on the engine VM(and there it fails): ovirt-log-collector --conf /rot/ovirt-log-collector.conf The failure comes right after 'add_secondary_storage_domains'[3] test, which all of its steps ran successfully. Can anyone look into this? Thanks, Nadav. [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc24-x86_... [2] https://github.com/oVirt/ovirt-system-tests/blob/master/basic_suite_master/t... [3] https://github.com/oVirt/ovirt-system-tests/blob/master/basic_suite_master/t... On Tue, Sep 20, 2016 at 9:45 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Fri, Sep 9, 2016 at 1:19 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
Indeed, this is the log collector. I wonder if we collect its logs... Y.
This can't be log-collector, it can be sos vdsm plugin. That said, if we run log-collector within lago we should collect the results as job artifacts.
On Thu, Sep 8, 2016 at 6:54 PM, Eyal Edri <eedri@redhat.com> wrote:
I'm pretty sure lago or ovirt system tests aren't doing it but its the log collector which is running during that test, I'm not near a computer so can't verify it yet.
On Sep 8, 2016 6:05 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
This looks like a lago issue - it should never read anything inside /rhev
This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists.
In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it.
Nir
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experi...
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel