4:26 a.m.
Failed test: basic_suite_master/002_bootstrap
Version: oVirt master
Link to failed job (Jenkins): ovirt-master_change-queue-tester/1817/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/>
Link to logs (Jenkins): link
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/artifa...
Suspected patch: Gerrit 80481/10 <https://gerrit.ovirt.org/#/c/80481/10>
Error snippet from logs:
*From host0*
MainThread::DEBUG::2017-08-17
05:03:20,501::cmd::63::root::(exec_sync_bytes) FAILED: <err> = '';
<rc> = 1
MainThread::ERROR::2017-08-17
05:03:20,502::initializer::53::root::(_lldp_init) Failed to enable
LLDP on eth0
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py",
line 51, in _lldp_init
Lldp.enable_lldp_on_iface(device)
File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py",
line 30, in enable_lldp_on_iface
lldptool.enable_lldp_on_iface(iface, rx_only)
File "/usr/lib/python2.7/site-packages/vdsm/network/lldpad/lldptool.py",
line 46, in enable_lldp_on_iface
raise EnableLldpError(rc, out, err, iface)
EnableLldpError: (1,
"timeout\n'M00000001C3040000000c04eth0000badminStatus0002rx' command
timed out.\n", '', 'eth0')
Thanks,
--
DANIEL BELENKY
RHV DEVOPS
EMEA VIRTUALIZATION R&D
<https://red.ht/sig>
5:29 a.m.
From my point of view the snippet from logs does not point to the
reason to fail the test.
On Thu, 17 Aug 2017 12:26:23 +0300
Daniel Belenky <dbelenky(a)redhat.com> wrote:
Failed test: basic_suite_master/002_bootstrap
Version: oVirt master
Link to failed job (Jenkins): ovirt-master_change-queue-tester/1817/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/>
Link to logs (Jenkins): link
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/artifa...
Suspected patch: Gerrit 80481/10
<https://gerrit.ovirt.org/#/c/80481/10>
Error snippet from logs:
*From host0*
MainThread::DEBUG::2017-08-17
05:03:20,501::cmd::63::root::(exec_sync_bytes) FAILED: <err> = '';
<rc> = 1
MainThread::ERROR::2017-08-17
05:03:20,502::initializer::53::root::(_lldp_init) Failed to enable
LLDP on eth0
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py",
line 51, in _lldp_init
Lldp.enable_lldp_on_iface(device)
File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py",
line 30, in enable_lldp_on_iface
lldptool.enable_lldp_on_iface(iface, rx_only)
File
"/usr/lib/python2.7/site-packages/vdsm/network/lldpad/lldptool.py",
line 46, in enable_lldp_on_iface raise EnableLldpError(rc, out, err,
iface) EnableLldpError: (1,
"timeout\n'M00000001C3040000000c04eth0000badminStatus0002rx' command
timed out.\n", '', 'eth0')
This error is expected [1] and will by fixed by REL 7.4.1 [2].
This error just blocks collecting lldp information and should not
influence anything else.
[1]
https://bugzilla.redhat.com/show_bug.cgi?id=1472722
[2]
https://bugzilla.redhat.com/show_bug.cgi?id=1479767
6:28 a.m.
This is a multi-part message in MIME format.
--------------3067C91F8A26101021D62FD5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Daniel, the link below is for failed test
002_bootstrap.add_secondary_storage_domains - is that the correct link?
If so, the error is in node01 and from what I can see, there is a race
between creating the target for the secondary domain and the mount
command sent from the host.
The reason I think its a race is because on the last try to mount the
server we are getting a different error than timeout and the error
suggests that we wither did not create the target or did not change the
permission to the target in the nfs server.
If you follow the log in node01, it looks like the connectStorageServer
command is succeeding but the mount commands times out at first
MountError: (32, ';mount.nfs: Connection timed out\n')
and than we get a "no such file or directly from the server.
MountError: (32, ';mount.nfs: mounting
192.168.201.3:/exports/nfs/exported failed, reason given by server: No
such file or directory\n')
Here's the two last mount errors from the vdsm log.
http://pastebin.test.redhat.com/509066
Here is the link to the vdsm log itself:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/artifa...
Thanks,
Dafna
On 08/17/2017 01:29 PM, Dominik Holler wrote:
From my point of view the snippet from logs does not point to the
reason to fail the test.
On Thu, 17 Aug 2017 12:26:23 +0300
Daniel Belenky <dbelenky(a)redhat.com> wrote:
> Failed test: basic_suite_master/002_bootstrap
>
> Version: oVirt master
>
> Link to failed job (Jenkins): ovirt-master_change-queue-tester/1817/
> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/>
>
> Link to logs (Jenkins): link
>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/artifa...
>
> Suspected patch: Gerrit 80481/10
> <https://gerrit.ovirt.org/#/c/80481/10>
>
>
> Error snippet from logs:
>
> *From host0*
>
> MainThread::DEBUG::2017-08-17
> 05:03:20,501::cmd::63::root::(exec_sync_bytes) FAILED: <err> = '';
> <rc> = 1
> MainThread::ERROR::2017-08-17
> 05:03:20,502::initializer::53::root::(_lldp_init) Failed to enable
> LLDP on eth0
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py",
> line 51, in _lldp_init
> Lldp.enable_lldp_on_iface(device)
> File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py",
> line 30, in enable_lldp_on_iface
> lldptool.enable_lldp_on_iface(iface, rx_only)
> File
> "/usr/lib/python2.7/site-packages/vdsm/network/lldpad/lldptool.py",
> line 46, in enable_lldp_on_iface raise EnableLldpError(rc, out, err,
> iface) EnableLldpError: (1,
> "timeout\n'M00000001C3040000000c04eth0000badminStatus0002rx' command
> timed out.\n", '', 'eth0')
>
>
This error is expected [1] and will by fixed by REL 7.4.1 [2].
This error just blocks collecting lldp information and should not
influence anything else.
[1]
https://bugzilla.redhat.com/show_bug.cgi?id=1472722
[2]
https://bugzilla.redhat.com/show_bug.cgi?id=1479767
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
--------------3067C91F8A26101021D62FD5
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Daniel, the link below is for failed
test <span style="color: rgb(51, 51, 51); font-size: 14px;
font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none; white-space:
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration-style: initial; text-decoration-color: initial;
display: inline ! important; float:
none;">002_bootstrap.add_secondary_storage_domains
- is that the correct link? </span><br>
<span style="color: rgb(51, 51, 51); font-size: 14px; font-style:
normal; font-variant-ligatures: normal; font-variant-caps:
normal; letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal; word-spacing:
0px; -webkit-text-stroke-width: 0px; text-decoration-style:
initial; text-decoration-color: initial; display: inline !
important; float: none;"></span><br>
<span style="color: rgb(51, 51, 51); font-size: 14px; font-style:
normal; font-variant-ligatures: normal; font-variant-caps:
normal; letter-spacing: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal; word-spacing:
0px; -webkit-text-stroke-width: 0px; text-decoration-style:
initial; text-decoration-color: initial; display: inline !
important; float: none;">If so, the error is in node01 and from
what I can see, there is a race between creating the target for
the secondary domain and the mount command sent from the host. <br>
The reason I think its a race is because on the last try to
mount the server we are getting a different error than timeout
and the error suggests that we wither did not create the target
or did not change the permission to the target in the nfs
server. <br>
<br>
If you follow the log in node01, it looks like the
connectStorageServer command is succeeding but the mount
commands times out at first <br>
<br>
MountError: (32, ';mount.nfs: Connection timed out\n')<br>
<br>
and than we get a "no such file or directly from the server. <br>
<br>
MountError: (32, ';mount.nfs: mounting
192.168.201.3:/exports/nfs/exported failed, reason given by
server: No such file or directory\n')<br>
<br>
Here's the two last mount errors from the vdsm log. <br>
<br>
<a class="moz-txt-link-freetext"
href="http://pastebin.test.redhat.com/509066">http://pastebi...
<br>
Here is the link to the vdsm log itself:
<a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
<br>
Thanks, <br>
Dafna</span><br>
<br>
On 08/17/2017 01:29 PM, Dominik Holler wrote:<br>
</div>
<blockquote type="cite"
cite="mid:20170817122949.68cad062@t460p">
<pre wrap="">From my point of view the snippet from logs does not
point to the
reason to fail the test.
On Thu, 17 Aug 2017 12:26:23 +0300
Daniel Belenky <a class="moz-txt-link-rfc2396E"
href="mailto:dbelenky@redhat.com"><dbelenky@redhat.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Failed test: basic_suite_master/002_bootstrap
Version: oVirt master
Link to failed job (Jenkins): ovirt-master_change-queue-tester/1817/
<a class="moz-txt-link-rfc2396E"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
Link to logs (Jenkins): link
<a class="moz-txt-link-rfc2396E"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
Suspected patch: Gerrit 80481/10
<a class="moz-txt-link-rfc2396E"
href="https://gerrit.ovirt.org/#/c/80481/10"><https:/...
Error snippet from logs:
*From host0*
MainThread::DEBUG::2017-08-17
05:03:20,501::cmd::63::root::(exec_sync_bytes) FAILED: <err> = '';
<rc> = 1
MainThread::ERROR::2017-08-17
05:03:20,502::initializer::53::root::(_lldp_init) Failed to enable
LLDP on eth0
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py",
line 51, in _lldp_init
Lldp.enable_lldp_on_iface(device)
File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py",
line 30, in enable_lldp_on_iface
lldptool.enable_lldp_on_iface(iface, rx_only)
File
"/usr/lib/python2.7/site-packages/vdsm/network/lldpad/lldptool.py",
line 46, in enable_lldp_on_iface raise EnableLldpError(rc, out, err,
iface) EnableLldpError: (1,
"timeout\n'M00000001C3040000000c04eth0000badminStatus0002rx' command
timed out.\n", '', 'eth0')
</pre>
</blockquote>
<pre wrap="">
This error is expected [1] and will by fixed by REL 7.4.1 [2].
This error just blocks collecting lldp information and should not
influence anything else.
[1]
<a class="moz-txt-link-freetext"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1472722">h...
[2]
<a class="moz-txt-link-freetext"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1479767">h...
_______________________________________________
Devel mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Devel@ovirt.org">Devel@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/devel">http://...
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------3067C91F8A26101021D62FD5--
6:08 a.m.
On Thu, Aug 17, 2017 at 12:26 PM, Daniel Belenky <dbelenky(a)redhat.com>
wrote:
Failed test: basic_suite_master/002_bootstrap
Version: oVirt master
Link to failed job (Jenkins): ovirt-master_change-queue-tester/1817/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/>
Link to logs (Jenkins): link
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1817/artifa...
Suspected patch: Gerrit 80481/10 <https://gerrit.ovirt.org/#/c/80481/10>
Error snippet from logs:
*From host0*
MainThread::DEBUG::2017-08-17 05:03:20,501::cmd::63::root::(exec_sync_bytes) FAILED:
<err> = ''; <rc> = 1
MainThread::ERROR::2017-08-17 05:03:20,502::initializer::53::root::(_lldp_init) Failed to
enable LLDP on eth0
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py", line 51,
in _lldp_init
Lldp.enable_lldp_on_iface(device)
File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py", line 30,
in enable_lldp_on_iface
lldptool.enable_lldp_on_iface(iface, rx_only)
File "/usr/lib/python2.7/site-packages/vdsm/network/lldpad/lldptool.py", line
46, in enable_lldp_on_iface
raise EnableLldpError(rc, out, err, iface)
EnableLldpError: (1,
"timeout\n'M00000001C3040000000c04eth0000badminStatus0002rx' command timed
out.\n", '', 'eth0')
where is this coming from? I'm seeing issues in host1 (vdsm log) :
2017-08-17 05:11:32,476-0400 ERROR (jsonrpc/6) [storage.HSM] Could not
connect to storageServer (hsm:2351)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2348,
in connectStorageServer
conObj.connect()
File "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py",
line 381, in connect
return self._mountCon.connect()
File "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py",
line 163, in connect
six.reraise(t, v, tb)
File "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py",
line 155, in connect
self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)
File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 208,
in mount
timeout=timeout, cgroup=cgroup)
File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in
__call__
return callMethod()
File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in
<lambda>
**kwargs)
File "<string>", line 2, in mount
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
MountError: (32, ';mount.nfs: mounting 192.168.201.3:/exports/nfs/exported
failed, reason given by server: No such file or directory\n')
Y.
Thanks,
--
DANIEL BELENKY
RHV DEVOPS
EMEA VIRTUALIZATION R&D
<https://red.ht/sig>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
2683
days inactive
2683
days old
3 comments
4 participants
participants (4)
-
Dafna Ron -
Daniel Belenky -
Dominik Holler -
Yaniv Kaul