3:08 p.m.
Hi all,
We had 2 issues causing vdsm check-patch and check-merge jobs to get stuck.
I fixed the one that caused most trouble:
https://gerrit.ovirt.org/57993
The other issue may be related to ioprocess, I fixed a related issue:
https://gerrit.ovirt.org/57473
But I have seen stuck jobs after this change, so the issue may not
be fixed yet.
If you see a stuck vdsm job - job that run more than 15 minutes, please
get me a backtrace:
1. locate the test_runner process pid:
$ ps aux | grep testrunner.py | grep -v grep
nsoffer 26297 82.6 0.9 389592 111144 pts/3 R+ 22:52 0:02
/usr/bin/python ../tests/testrunner.py ...
2. save a backtrace:
gdb attach 26297 --batch -ex "thread apply all py-bt" > py-bt.out
Thanks,
Nir
6:07 p.m.
On Thu, May 26, 2016 at 11:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
Hi all,
We had 2 issues causing vdsm check-patch and check-merge jobs to get stuck.
I fixed the one that caused most trouble:
https://gerrit.ovirt.org/57993
The other issue may be related to ioprocess, I fixed a related issue:
https://gerrit.ovirt.org/57473
But I have seen stuck jobs after this change, so the issue may not
be fixed yet.
If you see a stuck vdsm job - job that run more than 15 minutes, please
get me a backtrace:
1. locate the test_runner process pid:
$ ps aux | grep testrunner.py | grep -v grep
nsoffer 26297 82.6 0.9 389592 111144 pts/3 R+ 22:52 0:02
/usr/bin/python ../tests/testrunner.py ...
2. save a backtrace:
gdb attach 26297 --batch -ex "thread apply all py-bt" > py-bt.out
This requires the python-debuginfo package, typically installed using:
dnf debuginfo-install python
I sent this patch, detecting stuck vdsm tests, printing a backtrace, and killing
the stuck process:
https://gerrit.ovirt.org/58212
It works, but we don't get a backtrace, since python-debuginfo is not installed
although I require it - probably we need to add the fedora-debug repository
to check-patch.repos. I tried to use the urls from /etc/yum.repos.d/fedora.repo,
but none of them work.
I will need help from infra to get it working.
Nir
6:10 p.m.
It look like this when tests times out:
23:04:44 miscTests.EventTests
23:04:44 testEmit OK
23:04:44 testEmitCallbackException OK
23:04:49 testEmitStale OK
23:04:49 testInstanceMethod OK
23:04:50 testInstanceMethodDead OK
23:04:55 testOneShot
23:04:55 ========================================================================
23:04:55 = Timeout completing tests - extracting stacktrace
=
23:04:55 ========================================================================
23:04:55
23:04:55 attach: No such file or directory.
23:04:55 [New LWP 7887]
23:04:55 [New LWP 7880]
23:04:55 [New LWP 7873]
23:04:55 [New LWP 7866]
23:04:55 [New LWP 7859]
23:04:55 [New LWP 7852]
23:04:55 [New LWP 7845]
23:04:55 [Thread debugging using libthread_db enabled]
23:04:55 Using host libthread_db library "/lib64/libthread_db.so.1".
23:04:56 0x00007f17f0a1fa82 in pthread_cond_timedwait@(a)GLIBC_2.3.2 ()
from /lib64/libpthread.so.0
23:04:56
23:04:56 Thread 8 (Thread 0x7f17df860700 (LWP 7845)):
23:04:56 Undefined command: "py-bt". Try "help".
23:04:56 OK
23:04:56 testUnregister
23:04:56 ========================================================================
23:04:56 = Aborting tests
=
23:04:56 ========================================================================
23:04:56 ../tests/run_tests_local.sh: line 35: 7743 Killed
"$PYTHON_EXE" ../tests/testrunner.py --local-modules $@
On Sun, May 29, 2016 at 2:07 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
On Thu, May 26, 2016 at 11:08 PM, Nir Soffer
<nsoffer(a)redhat.com> wrote:
> Hi all,
>
> We had 2 issues causing vdsm check-patch and check-merge jobs to get stuck.
>
> I fixed the one that caused most trouble:
> https://gerrit.ovirt.org/57993
>
> The other issue may be related to ioprocess, I fixed a related issue:
> https://gerrit.ovirt.org/57473
>
> But I have seen stuck jobs after this change, so the issue may not
> be fixed yet.
>
> If you see a stuck vdsm job - job that run more than 15 minutes, please
> get me a backtrace:
>
> 1. locate the test_runner process pid:
>
> $ ps aux | grep testrunner.py | grep -v grep
> nsoffer 26297 82.6 0.9 389592 111144 pts/3 R+ 22:52 0:02
> /usr/bin/python ../tests/testrunner.py ...
>
> 2. save a backtrace:
>
> gdb attach 26297 --batch -ex "thread apply all py-bt" > py-bt.out
This requires the python-debuginfo package, typically installed using:
dnf debuginfo-install python
I sent this patch, detecting stuck vdsm tests, printing a backtrace, and killing
the stuck process:
https://gerrit.ovirt.org/58212
It works, but we don't get a backtrace, since python-debuginfo is not installed
although I require it - probably we need to add the fedora-debug repository
to check-patch.repos. I tried to use the urls from /etc/yum.repos.d/fedora.repo,
but none of them work.
I will need help from infra to get it working.
Nir
6:24 p.m.
On Sun, May 29, 2016 at 2:10 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
It look like this when tests times out:
23:04:44 miscTests.EventTests
23:04:44 testEmit OK
23:04:44 testEmitCallbackException OK
23:04:49 testEmitStale OK
23:04:49 testInstanceMethod OK
23:04:50 testInstanceMethodDead OK
23:04:55 testOneShot
23:04:55 ========================================================================
23:04:55 = Timeout completing tests - extracting stacktrace
=
23:04:55 ========================================================================
23:04:55
23:04:55 attach: No such file or directory.
23:04:55 [New LWP 7887]
23:04:55 [New LWP 7880]
23:04:55 [New LWP 7873]
23:04:55 [New LWP 7866]
23:04:55 [New LWP 7859]
23:04:55 [New LWP 7852]
23:04:55 [New LWP 7845]
23:04:55 [Thread debugging using libthread_db enabled]
23:04:55 Using host libthread_db library "/lib64/libthread_db.so.1".
23:04:56 0x00007f17f0a1fa82 in pthread_cond_timedwait@(a)GLIBC_2.3.2 ()
from /lib64/libpthread.so.0
23:04:56
23:04:56 Thread 8 (Thread 0x7f17df860700 (LWP 7845)):
23:04:56 Undefined command: "py-bt". Try "help".
23:04:56 OK
23:04:56 testUnregister
23:04:56 ========================================================================
23:04:56 = Aborting tests
=
23:04:56 ========================================================================
23:04:56 ../tests/run_tests_local.sh: line 35: 7743 Killed
"$PYTHON_EXE" ../tests/testrunner.py --local-modules $@
On Sun, May 29, 2016 at 2:07 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> On Thu, May 26, 2016 at 11:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>> Hi all,
>>
>> We had 2 issues causing vdsm check-patch and check-merge jobs to get stuck.
>>
>> I fixed the one that caused most trouble:
>> https://gerrit.ovirt.org/57993
>>
>> The other issue may be related to ioprocess, I fixed a related issue:
>> https://gerrit.ovirt.org/57473
>>
>> But I have seen stuck jobs after this change, so the issue may not
>> be fixed yet.
>>
>> If you see a stuck vdsm job - job that run more than 15 minutes, please
>> get me a backtrace:
>>
>> 1. locate the test_runner process pid:
>>
>> $ ps aux | grep testrunner.py | grep -v grep
>> nsoffer 26297 82.6 0.9 389592 111144 pts/3 R+ 22:52 0:02
>> /usr/bin/python ../tests/testrunner.py ...
>>
>> 2. save a backtrace:
>>
>> gdb attach 26297 --batch -ex "thread apply all py-bt" >
py-bt.out
>
> This requires the python-debuginfo package, typically installed using:
>
> dnf debuginfo-install python
>
> I sent this patch, detecting stuck vdsm tests, printing a backtrace, and killing
> the stuck process:
> https://gerrit.ovirt.org/58212
>
> It works, but we don't get a backtrace, since python-debuginfo is not installed
> although I require it - probably we need to add the fedora-debug repository
> to check-patch.repos. I tried to use the urls from /etc/yum.repos.d/fedora.repo,
> but none of them work.
>
> I will need help from infra to get it working.
I sent also this patch, that should fix the issue on jenkins, but I
cannot test it on jenkins:
https://gerrit.ovirt.org/58213
Nir
2:46 a.m.
--SkvwRMAIpAhPCcCJ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On 05/29 02:24, Nir Soffer wrote:
On Sun, May 29, 2016 at 2:10 AM, Nir Soffer
<nsoffer(a)redhat.com> wrote:
> It look like this when tests times out:
>
> 23:04:44 miscTests.EventTests
> 23:04:44 testEmit =
OK
> 23:04:44 testEmitCallbackException
=
OK
> 23:04:49 testEmitStale
=
OK
> 23:04:49 testInstanceMethod
=
OK
> 23:04:50 testInstanceMethodDead
=
OK
> 23:04:55 testOneShot
> 23:04:55 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> 23:04:55 =3D Timeout completing tests - extracting
stacktrace
> =3D
> 23:04:55 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> 23:04:55
> 23:04:55 attach: No such file or directory.
> 23:04:55 [New LWP 7887]
> 23:04:55 [New LWP 7880]
> 23:04:55 [New LWP 7873]
> 23:04:55 [New LWP 7866]
> 23:04:55 [New LWP 7859]
> 23:04:55 [New LWP 7852]
> 23:04:55 [New LWP 7845]
> 23:04:55 [Thread debugging using libthread_db enabled]
> 23:04:55 Using host libthread_db library "/lib64/libthread_db.so.1".
> 23:04:56 0x00007f17f0a1fa82 in pthread_cond_timedwait@(a)GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> 23:04:56
> 23:04:56 Thread 8 (Thread 0x7f17df860700 (LWP 7845)):
> 23:04:56 Undefined command: "py-bt". Try "help".
> 23:04:56 OK
> 23:04:56 testUnregister
> 23:04:56 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> 23:04:56 =3D Aborting tests
> =3D
> 23:04:56 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> 23:04:56 ../tests/run_tests_local.sh: line 35: 7743 Killed
> "$PYTHON_EXE" ../tests/testrunner.py --local-modules $@
>
>
>
> On Sun, May 29, 2016 at 2:07 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>> On Thu, May 26, 2016 at 11:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrot=
e:
>>> Hi all,
>>>
>>> We had 2 issues causing vdsm check-patch and check-merge jobs to get =
stuck.
>>>
>>> I fixed the one that caused most trouble:
>>> https://gerrit.ovirt.org/57993
>>>
>>> The other issue may be related to ioprocess, I fixed a related issue:
>>> https://gerrit.ovirt.org/57473
>>>
>>> But I have seen stuck jobs after this change, so the issue may not
>>> be fixed yet.
>>>
>>> If you see a stuck vdsm job - job that run more than 15 minutes, plea=
se
>>> get me a backtrace:
>>>
>>> 1. locate the test_runner process pid:
>>>
>>> $ ps aux | grep testrunner.py | grep -v grep
>>> nsoffer 26297 82.6 0.9 389592 111144 pts/3 R+ 22:52 0:02
>>> /usr/bin/python ../tests/testrunner.py ...
>>>
>>> 2. save a backtrace:
>>>
>>> gdb attach 26297 --batch -ex "thread apply all py-bt" >
py-bt.out
>>
>> This requires the python-debuginfo package, typically installed using:
>>
>> dnf debuginfo-install python
>>
>> I sent this patch, detecting stuck vdsm tests, printing a backtrace, a=
nd
killing
>> the stuck process:
>> https://gerrit.ovirt.org/58212
>>
>> It works, but we don't get a backtrace, since python-debuginfo is not =
installed
>> although I require it - probably we need to add the
fedora-debug repos=
itory
>> to check-patch.repos. I tried to use the urls from
/etc/yum.repos.d/fe=
dora.repo,
>> but none of them work.
>>
>> I will need help from infra to get it working.
=20
I sent also this patch, that should fix the issue on jenkins, but I
cannot test it on jenkins:
https://gerrit.ovirt.org/58213
Instead of forcing adding the repo for all the projects, you should use the
*repos files that vdsm has in the automation directory to add there any ext=
ra
repos that you want when running/installing
=20
Nir
--=20
David Caro
Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D
Tel.: +420 532 294 605
Email: dcaro(a)redhat.com
IRC: dcaro|dcaroest@{freenode|oftc|redhat}
Web: www.redhat.com
RHT Global #: 82-62605
--SkvwRMAIpAhPCcCJ
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJXS+/nAAoJEEBxx+HSYmnDv1oH/jtM0tQ9G3j5Be3pUKRvB307
d8k9krvxfXKG8xaxZ+1qeMV0ix3Rlkj7o3Cm1GLbVdYd5mvsnbT86OGQMY0aRDKj
FLfFxOcbIsRgtM2DU6wQW8XujYlt6FD6WHDkIYSUlY7RddtXCLVteQP0rF8bwhvd
vu3QxI7/oK72GDoejfLkvB/XMU0NtXNCh/E9zCKJvZo9Wl+ORNJ6h8hmNYMPktUF
58dCxDhj25LGAFCokj/ILsTXQtTHiNEI++bTH4u557m089Hdz1HIQsZxvbrZOvd1
zlzJPuE2Yr+Szw0bxlAX+99gJNEklT+AZsDZQmnt/+GmWmAmGKrYdHRdI/WTLQc=
=PWhi
-----END PGP SIGNATURE-----
--SkvwRMAIpAhPCcCJ--
4:56 a.m.
On Mon, May 30, 2016 at 10:46 AM, David Caro Estevez <dcaro(a)redhat.com> wrote:
On 05/29 02:24, Nir Soffer wrote:
> On Sun, May 29, 2016 at 2:10 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> > It look like this when tests times out:
> >
> > 23:04:44 miscTests.EventTests
> > 23:04:44 testEmit OK
> > 23:04:44 testEmitCallbackException OK
> > 23:04:49 testEmitStale OK
> > 23:04:49 testInstanceMethod OK
> > 23:04:50 testInstanceMethodDead OK
> > 23:04:55 testOneShot
> > 23:04:55
========================================================================
> > 23:04:55 = Timeout completing tests - extracting stacktrace
> > =
> > 23:04:55
========================================================================
> > 23:04:55
> > 23:04:55 attach: No such file or directory.
> > 23:04:55 [New LWP 7887]
> > 23:04:55 [New LWP 7880]
> > 23:04:55 [New LWP 7873]
> > 23:04:55 [New LWP 7866]
> > 23:04:55 [New LWP 7859]
> > 23:04:55 [New LWP 7852]
> > 23:04:55 [New LWP 7845]
> > 23:04:55 [Thread debugging using libthread_db enabled]
> > 23:04:55 Using host libthread_db library "/lib64/libthread_db.so.1".
> > 23:04:56 0x00007f17f0a1fa82 in pthread_cond_timedwait@(a)GLIBC_2.3.2 ()
> > from /lib64/libpthread.so.0
> > 23:04:56
> > 23:04:56 Thread 8 (Thread 0x7f17df860700 (LWP 7845)):
> > 23:04:56 Undefined command: "py-bt". Try "help".
> > 23:04:56 OK
> > 23:04:56 testUnregister
> > 23:04:56
========================================================================
> > 23:04:56 = Aborting tests
> > =
> > 23:04:56
========================================================================
> > 23:04:56 ../tests/run_tests_local.sh: line 35: 7743 Killed
> > "$PYTHON_EXE" ../tests/testrunner.py --local-modules $@
> >
> >
> >
> > On Sun, May 29, 2016 at 2:07 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> >> On Thu, May 26, 2016 at 11:08 PM, Nir Soffer <nsoffer(a)redhat.com>
wrote:
> >>> Hi all,
> >>>
> >>> We had 2 issues causing vdsm check-patch and check-merge jobs to get
stuck.
> >>>
> >>> I fixed the one that caused most trouble:
> >>> https://gerrit.ovirt.org/57993
> >>>
> >>> The other issue may be related to ioprocess, I fixed a related issue:
> >>> https://gerrit.ovirt.org/57473
> >>>
> >>> But I have seen stuck jobs after this change, so the issue may not
> >>> be fixed yet.
> >>>
> >>> If you see a stuck vdsm job - job that run more than 15 minutes, please
> >>> get me a backtrace:
> >>>
> >>> 1. locate the test_runner process pid:
> >>>
> >>> $ ps aux | grep testrunner.py | grep -v grep
> >>> nsoffer 26297 82.6 0.9 389592 111144 pts/3 R+ 22:52 0:02
> >>> /usr/bin/python ../tests/testrunner.py ...
> >>>
> >>> 2. save a backtrace:
> >>>
> >>> gdb attach 26297 --batch -ex "thread apply all py-bt" >
py-bt.out
> >>
> >> This requires the python-debuginfo package, typically installed using:
> >>
> >> dnf debuginfo-install python
> >>
> >> I sent this patch, detecting stuck vdsm tests, printing a backtrace, and
killing
> >> the stuck process:
> >> https://gerrit.ovirt.org/58212
> >>
> >> It works, but we don't get a backtrace, since python-debuginfo is not
installed
> >> although I require it - probably we need to add the fedora-debug repository
> >> to check-patch.repos. I tried to use the urls from
/etc/yum.repos.d/fedora.repo,
> >> but none of them work.
> >>
> >> I will need help from infra to get it working.
>
> I sent also this patch, that should fix the issue on jenkins, but I
> cannot test it on jenkins:
> https://gerrit.ovirt.org/58213
Instead of forcing adding the repo for all the projects, you should use the
*repos files that vdsm has in the automation directory to add there any extra
repos that you want when running/installing
Ok, I'm using now debuginfo-install to install the package -
automatically enable
the debuginfo repo, and it works on fedora.
On el7, we fail to install the debuginfo package - I don't know why:
09:03:43 + debuginfo-install -y python
09:03:45 Could not find debuginfo for main pkg: python-2.7.5-34.el7.x86_64
09:03:45 Could not find debuginfo pkg for dependency package
glibc-2.17-106.el7_2.6.x86_64
09:03:45 Could not find debuginfo pkg for dependency package
python-libs-2.7.5-34.el7.x86_64
Maybe a missing repo?
On fc23, we get the debuginfo package and we get a backtrace:
09:06:05 testlibTests.TestStuckProcess
09:09:19 test_stuck
09:09:19 ========================================================================
09:09:19 = Timeout completing tests - extracting stacktrace
=
09:09:19 ========================================================================
09:09:19
09:09:19 attach: No such file or directory.
09:09:19 [New LWP 50476]
09:09:19 [New LWP 50469]
09:09:19 [New LWP 50462]
09:09:19 [New LWP 50455]
09:09:19 [New LWP 50448]
09:09:19 [New LWP 50441]
09:09:19 [New LWP 50434]
09:09:19 [New LWP 50427]
09:09:19 [New LWP 50420]
09:09:19 [New LWP 50413]
09:09:19 [New LWP 50406]
09:09:19 [New LWP 50399]
09:09:19 [New LWP 50392]
09:09:19 [New LWP 50385]
09:09:19 [New LWP 50378]
09:09:19 [New LWP 50371]
09:09:19 [New LWP 50364]
09:09:19 [New LWP 50247]
09:09:19 [New LWP 50240]
09:09:19 [New LWP 50233]
09:09:19 [New LWP 50226]
09:09:19 [New LWP 50219]
09:09:19 [New LWP 50216]
09:09:19 [New LWP 50189]
09:09:19 [New LWP 50175]
09:09:19 [New LWP 50161]
09:09:19 [New LWP 50147]
09:09:19 [New LWP 50133]
09:09:19 [New LWP 50119]
09:09:19 [New LWP 50105]
09:09:19 [New LWP 50091]
09:09:19 [New LWP 49889]
09:09:19 [New LWP 49858]
09:09:19 [Thread debugging using libthread_db enabled]
09:09:19 Using host libthread_db library "/lib64/libthread_db.so.1".
09:09:20 0x00007fb5ffece8a3 in select () at
../sysdeps/unix/syscall-template.S:84
09:09:20 84 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
09:09:20
09:09:20 Thread 34 (Thread 0x7fb5ab7fe700 (LWP 49858)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 33 (Thread 0x7fb5a9899700 (LWP 49889)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 32 (Thread 0x7fb58effd700 (LWP 50091)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 31 (Thread 0x7fb58e7fc700 (LWP 50105)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 30 (Thread 0x7fb58dffb700 (LWP 50119)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 29 (Thread 0x7fb58d7fa700 (LWP 50133)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 28 (Thread 0x7fb58cff9700 (LWP 50147)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 27 (Thread 0x7fb56bfff700 (LWP 50161)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 26 (Thread 0x7fb56b7fe700 (LWP 50175)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 25 (Thread 0x7fb56affd700 (LWP 50189)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 24 (Thread 0x7fb59affd700 (LWP 50216)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 23 (Thread 0x7fb5e905f700 (LWP 50219)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 22 (Thread 0x7fb59bfff700 (LWP 50226)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 21 (Thread 0x7fb5e885e700 (LWP 50233)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 20 (Thread 0x7fb5e9860700 (LWP 50240)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 19 (Thread 0x7fb5eb864700 (LWP 50247)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 18 (Thread 0x7fb5c9061700 (LWP 50364)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 17 (Thread 0x7fb5cb7fe700 (LWP 50371)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 16 (Thread 0x7fb5cbfff700 (LWP 50378)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 15 (Thread 0x7fb5ea862700 (LWP 50385)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 14 (Thread 0x7fb5eb063700 (LWP 50392)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 13 (Thread 0x7fb5aaffd700 (LWP 50399)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 12 (Thread 0x7fb5a8f18700 (LWP 50406)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 11 (Thread 0x7fb59b7fe700 (LWP 50413)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 10 (Thread 0x7fb59a7fc700 (LWP 50420)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 9 (Thread 0x7fb599ffb700 (LWP 50427)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 8 (Thread 0x7fb5997fa700 (LWP 50434)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 7 (Thread 0x7fb598ff9700 (LWP 50441)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 6 (Thread 0x7fb58ffff700 (LWP 50448)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 5 (Thread 0x7fb58f7fe700 (LWP 50455)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 4 (Thread 0x7fb56a7fc700 (LWP 50462)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 3 (Thread 0x7fb569ffb700 (LWP 50469)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 2 (Thread 0x7fb5697fa700 (LWP 50476)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 1 (Thread 0x7fb60109d700 (LWP 49417)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlibTests.py",
line 264, in test_stuck
09:09:20 time.sleep(300)
09:09:20 File "/usr/lib64/python2.7/unittest/case.py", line 367, in run
09:09:20 testMethod()
09:09:20 File "/usr/lib64/python2.7/unittest/case.py", line 431, in __call__
09:09:20 return self.run(*args, **kwds)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/case.py", line
151, in runTest
09:09:20 test(result)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/case.py", line
133, in run
09:09:20 self.runTest(result)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/case.py", line
45, in __call__
09:09:20 return self.run(*arg, **kwarg)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line 62, in
run
09:09:20 test(result)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line
207, in runTests
09:09:20 result = self.testRunner.run(self.test)
09:09:20 File "/usr/lib64/python2.7/unittest/main.py", line 95, in __init__
09:09:20 self.runTests()
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line
121, in __init__
09:09:20 **extra_args)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line
301, in run
09:09:20 return TestProgram(*arg, **kw).success
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py",
line 396, in run
09:09:20 sys.exit(not core.run(config=conf, testRunner=runner, argv=argv))
09:09:20 File "../tests/testrunner.py", line 88, in <module>
09:09:20 testlib.run()
09:09:20
09:09:20 ========================================================================
09:09:20 = Aborting tests
=
09:09:20 ========================================================================
>
> Nir
--
David Caro
Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D
Tel.: +420 532 294 605
Email: dcaro(a)redhat.com
IRC: dcaro|dcaroest@{freenode|oftc|redhat}
Web: www.redhat.com
RHT Global #: 82-62605
2880
days inactive
2886
days old
5 comments
2 participants
participants (2)
-
David Caro Estevez -
Nir Soffer