Re: [ovirt-devel] test-repo_ovirt_experimental_master job - failed
Adding devel. On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d- 9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/ eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_ master/ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/ test-repo_ovirt_experimental_master/1462/testReport/(root)/ 002_bootstrap/add_secondary_storage_domains/
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
This looks like a lago issue - it should never read anything inside /rhev This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists. In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it. Nir
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experi...
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
I'm pretty sure lago or ovirt system tests aren't doing it but its the log collector which is running during that test, I'm not near a computer so can't verify it yet. On Sep 8, 2016 6:05 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-
9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp:
No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d- 9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
This looks like a lago issue - it should never read anything inside /rhev
This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists.
In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it.
Nir
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_ master/ovirt-system-tests/basic_suite_master/test- scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/test- repo_ovirt_experimental_master/1462/testReport/(root)/ 002_bootstrap/add_secondary_storage_domains/
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Indeed, this is the log collector. I wonder if we collect its logs... Y. On Thu, Sep 8, 2016 at 6:54 PM, Eyal Edri <eedri@redhat.com> wrote:
I'm pretty sure lago or ovirt system tests aren't doing it but its the log collector which is running during that test, I'm not near a computer so can't verify it yet.
On Sep 8, 2016 6:05 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-945189
0fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp:
No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-945189 0fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
This looks like a lago issue - it should never read anything inside /rhev
This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists.
In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it.
Nir
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master /ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/test-r epo_ovirt_experimental_master/1462/testReport/(root)/002_ bootstrap/add_secondary_storage_domains/
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Fri, Sep 9, 2016 at 1:19 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
Indeed, this is the log collector. I wonder if we collect its logs... Y.
This can't be log-collector, it can be sos vdsm plugin. That said, if we run log-collector within lago we should collect the results as job artifacts.
On Thu, Sep 8, 2016 at 6:54 PM, Eyal Edri <eedri@redhat.com> wrote:
I'm pretty sure lago or ovirt system tests aren't doing it but its the log collector which is running during that test, I'm not near a computer so can't verify it yet.
On Sep 8, 2016 6:05 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-945189
0fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp:
No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-945189 0fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
This looks like a lago issue - it should never read anything inside /rhev
This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists.
In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it.
Nir
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master /ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/test-r epo_ovirt_experimental_master/1462/testReport/(root)/002_boo tstrap/add_secondary_storage_domains/
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
Hi all, bumping this thread due to an almost identical failure[1]: ovirt-log-collector/ovirt-log-collector-20161030053238.log:2016-10-30 05:33:09::ERROR::__main__::791::root:: Failed to collect logs from: 192.168.200.4; /bin/ls: /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.job.0: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.recover.1: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.task: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5-5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7-52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.recover.0: No such file or directory To ensure I've checked lago/OST, and couldn't find any stage where there is a reference to '/rhv' nor any manipulation to ovirt-log-collector, only customizations made is a 'ovirt-log-collector.conf' with user/password. The code that pulls the logs in OST[2] runs the following command on the engine VM(and there it fails): ovirt-log-collector --conf /rot/ovirt-log-collector.conf The failure comes right after 'add_secondary_storage_domains'[3] test, which all of its steps ran successfully. Can anyone look into this? Thanks, Nadav. [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc24-x86_... [2] https://github.com/oVirt/ovirt-system-tests/blob/master/basic_suite_master/t... [3] https://github.com/oVirt/ovirt-system-tests/blob/master/basic_suite_master/t... On Tue, Sep 20, 2016 at 9:45 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Fri, Sep 9, 2016 at 1:19 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
Indeed, this is the log collector. I wonder if we collect its logs... Y.
This can't be log-collector, it can be sos vdsm plugin. That said, if we run log-collector within lago we should collect the results as job artifacts.
On Thu, Sep 8, 2016 at 6:54 PM, Eyal Edri <eedri@redhat.com> wrote:
I'm pretty sure lago or ovirt system tests aren't doing it but its the log collector which is running during that test, I'm not near a computer so can't verify it yet.
On Sep 8, 2016 6:05 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <sbendavi@redhat.com> wrote:
Hi,
Job [1] is failing with the following error:
lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine errors: ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d-9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: No such file or directory
This looks like a lago issue - it should never read anything inside /rhev
This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists.
In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it.
Nir
lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in _ret_via_queue queue.put({'return': func()}) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 493, in log_collector result.code, 0, 'log collector failed. Exit code is %s' % result.code File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) AssertionError: log collector failed. Exit code is 2
* The previous issue already fixed (SDK) and now we have a new issue on the same area.
[1] - http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experi...
Best Regards,
Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL RHCSA | RHCE IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Sun, Oct 30, 2016 at 12:26 PM, Nadav Goldin <ngoldin@redhat.com> wrote:
Hi all, bumping this thread due to an almost identical failure[1]:
ovirt-log-collector/ovirt-log-collector-20161030053238.log:2016-10-30 05:33:09::ERROR::__main__::791::root:: Failed to collect logs from: 192.168.200.4; /bin/ls: /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5- 5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7- 52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.job.0: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5- 5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7- 52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.recover.1: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5- 5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7- 52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.task: No such file or directory ovirt-log-collector/ovirt-log-collector-20161030053238.log-/bin/ls: cannot access /rhev/data-center/mnt/blockSD/63c4fdd3-5d0f-4d16-b1e5- 5f43caa4cf82/master/tasks/6b3b6aa1-808c-42df-9db7- 52349f8533f2/6b3b6aa1-808c-42df-9db7-52349f8533f2.recover.0: No such file or directory
To ensure I've checked lago/OST, and couldn't find any stage where there is a reference to '/rhv' nor any manipulation to ovirt-log-collector, only customizations made is a 'ovirt-log-collector.conf' with user/password. The code that pulls the logs in OST[2] runs the following command on the engine VM(and there it fails):
ovirt-log-collector --conf /rot/ovirt-log-collector.conf
The failure comes right after 'add_secondary_storage_domains'[3] test, which all of its steps ran successfully.
Not exactly.
Can anyone look into this?
It may be my fault, in a way. I've added the log collector test to run in parallel to the tests that add the secondary storage domains. The directories it tries to access may or may not be available - this is probably racy. I don't think it should fail, but I can certainly see why it can. The easiest 'fix' would be to split it to its own test (I wanted to save execution time, as most of the time spent on secondary storage domains test is not really useful). Y.
Thanks, Nadav.
[1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_ check-patch-fc24-x86_64/141/console [2] https://github.com/oVirt/ovirt-system-tests/blob/ master/basic_suite_master/test-scenarios/002_bootstrap.py#L490 [3] https://github.com/oVirt/ovirt-system-tests/blob/ master/basic_suite_master/test-scenarios/002_bootstrap.py#L243
On Tue, Sep 20, 2016 at 9:45 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Fri, Sep 9, 2016 at 1:19 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
Indeed, this is the log collector. I wonder if we collect its logs... Y.
This can't be log-collector, it can be sos vdsm plugin. That said, if we run log-collector within lago we should collect the
results as job artifacts.
On Thu, Sep 8, 2016 at 6:54 PM, Eyal Edri <eedri@redhat.com> wrote:
I'm pretty sure lago or ovirt system tests aren't doing it but its the
On Sep 8, 2016 6:05 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
On Thu, Sep 8, 2016 at 5:45 PM, Eyal Edri <eedri@redhat.com> wrote:
Adding devel.
On Thu, Sep 8, 2016 at 5:43 PM, Shlomo Ben David <
sbendavi@redhat.com>
wrote: > > Hi, > > Job [1] is failing with the following error: > > lago.ssh: DEBUG: Command 8de75538 on lago_basic_suite_master_engine > errors: > ERROR: Failed to collect logs from: 192.168.200.2; /bin/ls: > /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d- 9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: > No such file or directory > /bin/ls: cannot open directory > /rhev/data-center/mnt/blockSD/eb8c9f48-5f23-48dc-ab7d- 9451890fd422/master/tasks/1350bed7-443e-4ae6-ae1f-9b24d18c70a8.temp: > No such file or directory
This looks like a lago issue - it should never read anything inside /rhev
This is a private directory for vdsm, no other process should ever depend on the content inside this directory, or even on the fact that it exists.
In particular, /rhev/data-center/mnt/blockSD/*/master/tasks/*.temp Is not a log file, and lago should not collect it.
Nir
> lago.utils: ERROR: Error while running thread > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/lago/utils.py", line 53, in > _ret_via_queue > queue.put({'return': func()}) > File > "/home/jenkins/workspace/test-repo_ovirt_experimental_ master/ovirt-system-tests/basic_suite_master/test- scenarios/002_bootstrap.py", > line 493, in log_collector > result.code, 0, 'log collector failed. Exit code is %s' % result.code > File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py",
log collector which is running during that test, I'm not near a computer so can't verify it yet. line 29,
> in eq_ > raise AssertionError(msg or "%r != %r" % (a, b)) > AssertionError: log collector failed. Exit code is 2 > > > * The previous issue already fixed (SDK) and now we have a new issue on > the same area. > > > [1] - > http://jenkins.ovirt.org/view/experimental%20jobs/job/test- repo_ovirt_experimental_master/1462/testReport/(root)/ 002_bootstrap/add_secondary_storage_domains/ > > > Best Regards, > > Shlomi Ben-David | DevOps Engineer | Red Hat ISRAEL > RHCSA | RHCE > IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci) > > OPEN SOURCE - 1 4 011 && 011 4 1
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Sun, Oct 30, 2016 at 12:40 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
Not exactly.
My bad, missed that the tests run in parallel, though what this means is that 'ovirt-log-collector' can fail when there are ongoing tasks(such as adding the storage domains), I assume that is not the expected behaviour. I'll send a patch separating the test for now.
On Sun, Oct 30, 2016 at 12:57 PM, Nadav Goldin <ngoldin@redhat.com> wrote:
On Sun, Oct 30, 2016 at 12:40 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
Not exactly.
My bad, missed that the tests run in parallel, though what this means is that 'ovirt-log-collector' can fail when there are ongoing tasks(such as adding the storage domains), I assume that is not the expected behaviour. I'll send a patch separating the test for now.
participants (5)
-
Eyal Edri -
Nadav Goldin -
Nir Soffer -
Sandro Bonazzola -
Yaniv Kaul