[VDSM] stuck tests in the ci - update
by Nir Soffer
Hi all,
So we have this patch, aborting stuck tests and printing a backtrace
of all threads:
https://gerrit.ovirt.org/#/c/58212
On fedora, we get now a nice backtrace (see bellow) - which show that
we have unexpected
threads running during the tests.
we have 25 iopproces threads like this:
09:09:20 Thread 34 (Thread 0x7fb5ab7fe700 (LWP 49858)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
This is expected given the current code managing these, and are daemon threads.
We have 8 reactor threads:
09:09:20 Thread 32 (Thread 0x7fb58effd700 (LWP 50091)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
I don't know where these are started and it they are daemon threads.
Searching the tests, we have too much code creating threads directly,
instead of using vdsm.concurrent.thread:
$ git grep Thread tests/ | wc -l
79
Please check your tests, and replace all threads to use testlib.start_thread()
creating daemon threads.
Code inheriting from threading.Thread is the worst, please keep a thread
instance instead, and start it with testlib.start_thread().
I sent this patch fixing sslTests for now:
https://gerrit.ovirt.org/58435
Nir
----
09:06:05 testlibTests.TestStuckProcess
09:09:19 test_stuck
09:09:19 ========================================================================
09:09:19 = Timeout completing tests - extracting stacktrace
=
09:09:19 ========================================================================
09:09:19
09:09:19 attach: No such file or directory.
09:09:19 [New LWP 50476]
09:09:19 [New LWP 50469]
09:09:19 [New LWP 50462]
09:09:19 [New LWP 50455]
09:09:19 [New LWP 50448]
09:09:19 [New LWP 50441]
09:09:19 [New LWP 50434]
09:09:19 [New LWP 50427]
09:09:19 [New LWP 50420]
09:09:19 [New LWP 50413]
09:09:19 [New LWP 50406]
09:09:19 [New LWP 50399]
09:09:19 [New LWP 50392]
09:09:19 [New LWP 50385]
09:09:19 [New LWP 50378]
09:09:19 [New LWP 50371]
09:09:19 [New LWP 50364]
09:09:19 [New LWP 50247]
09:09:19 [New LWP 50240]
09:09:19 [New LWP 50233]
09:09:19 [New LWP 50226]
09:09:19 [New LWP 50219]
09:09:19 [New LWP 50216]
09:09:19 [New LWP 50189]
09:09:19 [New LWP 50175]
09:09:19 [New LWP 50161]
09:09:19 [New LWP 50147]
09:09:19 [New LWP 50133]
09:09:19 [New LWP 50119]
09:09:19 [New LWP 50105]
09:09:19 [New LWP 50091]
09:09:19 [New LWP 49889]
09:09:19 [New LWP 49858]
09:09:19 [Thread debugging using libthread_db enabled]
09:09:19 Using host libthread_db library "/lib64/libthread_db.so.1".
09:09:20 0x00007fb5ffece8a3 in select () at
../sysdeps/unix/syscall-template.S:84
09:09:20 84 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
09:09:20
09:09:20 Thread 34 (Thread 0x7fb5ab7fe700 (LWP 49858)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 33 (Thread 0x7fb5a9899700 (LWP 49889)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 32 (Thread 0x7fb58effd700 (LWP 50091)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 31 (Thread 0x7fb58e7fc700 (LWP 50105)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 30 (Thread 0x7fb58dffb700 (LWP 50119)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 29 (Thread 0x7fb58d7fa700 (LWP 50133)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 28 (Thread 0x7fb58cff9700 (LWP 50147)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 27 (Thread 0x7fb56bfff700 (LWP 50161)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 26 (Thread 0x7fb56b7fe700 (LWP 50175)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 25 (Thread 0x7fb56affd700 (LWP 50189)):
09:09:20 Traceback (most recent call first):
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 192, in poll2
09:09:20 r = pollster.poll(timeout)
09:09:20 File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
09:09:20 poll_fun(timeout, map)
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 212, in process_requests
09:09:20 count=1,
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 24 (Thread 0x7fb59affd700 (LWP 50216)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 23 (Thread 0x7fb5e905f700 (LWP 50219)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 22 (Thread 0x7fb59bfff700 (LWP 50226)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 21 (Thread 0x7fb5e885e700 (LWP 50233)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 20 (Thread 0x7fb5e9860700 (LWP 50240)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 19 (Thread 0x7fb5eb864700 (LWP 50247)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 18 (Thread 0x7fb5c9061700 (LWP 50364)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 17 (Thread 0x7fb5cb7fe700 (LWP 50371)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 16 (Thread 0x7fb5cbfff700 (LWP 50378)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 15 (Thread 0x7fb5ea862700 (LWP 50385)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 14 (Thread 0x7fb5eb063700 (LWP 50392)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 13 (Thread 0x7fb5aaffd700 (LWP 50399)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 12 (Thread 0x7fb5a8f18700 (LWP 50406)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 11 (Thread 0x7fb59b7fe700 (LWP 50413)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 10 (Thread 0x7fb59a7fc700 (LWP 50420)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 9 (Thread 0x7fb599ffb700 (LWP 50427)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 8 (Thread 0x7fb5997fa700 (LWP 50434)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 7 (Thread 0x7fb598ff9700 (LWP 50441)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 6 (Thread 0x7fb58ffff700 (LWP 50448)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 5 (Thread 0x7fb58f7fe700 (LWP 50455)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 4 (Thread 0x7fb56a7fc700 (LWP 50462)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 3 (Thread 0x7fb569ffb700 (LWP 50469)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 2 (Thread 0x7fb5697fa700 (LWP 50476)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 222, in
NoIntrPoll
09:09:20 return pollfun(timeout * 1000) # timeout for poll is in ms
09:09:20 File
"/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 115, in
_communicate
09:09:20 pollres = NoIntrPoll(poller.poll, 5)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 757, in run
09:09:20 self.__target(*self.__args, **self.__kwargs)
09:09:20 File "/usr/lib64/python2.7/threading.py", line 804, in
__bootstrap_inner
09:09:20 self.run()
09:09:20 File "/usr/lib64/python2.7/threading.py", line 777, in __bootstrap
09:09:20 self.__bootstrap_inner()
09:09:20
09:09:20 Thread 1 (Thread 0x7fb60109d700 (LWP 49417)):
09:09:20 Traceback (most recent call first):
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlibTests.py",
line 264, in test_stuck
09:09:20 time.sleep(300)
09:09:20 File "/usr/lib64/python2.7/unittest/case.py", line 367, in run
09:09:20 testMethod()
09:09:20 File "/usr/lib64/python2.7/unittest/case.py", line 431, in __call__
09:09:20 return self.run(*args, **kwds)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/case.py", line
151, in runTest
09:09:20 test(result)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/case.py", line
133, in run
09:09:20 self.runTest(result)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/case.py", line
45, in __call__
09:09:20 return self.run(*arg, **kwarg)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
224, in run
09:09:20 test(orig)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/suite.py", line
177, in __call__
09:09:20 return self.run(*arg, **kw)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line 62, in run
09:09:20 test(result)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line
207, in runTests
09:09:20 result = self.testRunner.run(self.test)
09:09:20 File "/usr/lib64/python2.7/unittest/main.py", line 95, in __init__
09:09:20 self.runTests()
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line
121, in __init__
09:09:20 **extra_args)
09:09:20 File "/usr/lib/python2.7/site-packages/nose/core.py", line
301, in run
09:09:20 return TestProgram(*arg, **kw).success
09:09:20 File
"/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py",
line 396, in run
09:09:20 sys.exit(not core.run(config=conf, testRunner=runner, argv=argv))
09:09:20 File "../tests/testrunner.py", line 88, in <module>
09:09:20 testlib.run()
09:09:20
09:09:20 ========================================================================
09:09:20 = Aborting tests
=
09:09:20 ========================================================================
7 years, 11 months
- 2
- 1
[ACTION REQUESTED] i18n default English text is now stored in properties files
by Scott Dickerson
Hi,
In order to resolve bug [1] and prep [2], the default English text for I18N
Constants and Messages have been moved to their corresponding properties
files. Going forward, if a new constant or message needs to be added,
please add the key method to the proper interface and then add the key plus
English text to the interface's corresponding properties file. Checks will
be added, in the next few days, to fail the build process if the GWT
annotations (@DefaultStringValue, @DefaultMessage) are used.
This change has been made to simplify the translation process [1] and to
prepare for the replacement of the GWT i18n engine in 4.1 [2].
For example, in the webadmin project, the ApplicationConstants [3]
interface previously looked like:
package org.ovirt.engine.ui.webadmin;
import org.ovirt.engine.ui.common.CommonApplicationConstants;
public interface ApplicationConstants extends CommonApplicationConstants
{
@DefaultStringValue("oVirt Engine Web Administration")
String applicationTitle();
@DefaultStringValue("About")
String aboutPopupCaption();
@DefaultStringValue("oVirt Engine Version:")
String ovirtVersionAbout();
Now, the interface looks like this [4]:
package org.ovirt.engine.ui.webadmin;
import org.ovirt.engine.ui.common.CommonApplicationConstants;
public interface ApplicationConstants extends CommonApplicationConstants
{
String applicationTitle();
String aboutPopupCaption();
String ovirtVersionAbout();
With a new properties file created, ApplicationContants.properties [5],
with this kind of content :
#
#
# Bundle name: org.ovirt.engine.ui.webadmin.ApplicationConstants
#
#
#Thu May 19 01:24:21 EDT 2016
aboutPopupCaption=About
applicationTitle=oVirt Engine Web Administration
ovirtVersionAbout=oVirt Engine Version\:
Let me know if there are any questions.
Regards,
Scott
--
Scott Dickerson
Senior Software Engineer
RHEV-M Engineering - UX Team
Red Hat, Inc
[1] - https://bugzilla.redhat.com/show_bug.cgi?id=1224423
[2] - https://bugzilla.redhat.com/show_bug.cgi?id=1287408
[3] -
https://github.com/oVirt/ovirt-engine/blob/569695916d2325d0c1e1fdabb6f85c...
[4] -
https://github.com/oVirt/ovirt-engine/blob/master/frontend/webadmin/modul...
[5] -
https://github.com/oVirt/ovirt-engine/blob/master/frontend/webadmin/modul...
7 years, 11 months
- 4
- 6
Interacting with VDSM storage without oVirt Engine
by Hayley Swimelar
Hello all,
Like the title says, I need to work with VDSM's storage layer without
involving the engine.
Presently, I'm testing this with NFS domains and have been able to use
vdsClient to create, attach, and activate a new storage domain, but I
have not been able to create a new image or volume.
Here are the commands I ran to get to this step, starting with a nfs
volume already mounted on the machine I ran the commands on.
----
vdsClient -s host_name createStorageDomain 1
97338d5c-9b6f-1859-b827-e977ed082e53 cli_domain storage_server:/data
vdsClient -s host_name attachStorageDomain
97338d5c-9b6f-1859-b827-e977ed082e53 00000001-0001-0001-0001-00000000033c
vdsClient -s host_name activateStorageDomain
97338d5c-9b6f-1859-b827-e977ed082e53 00000001-0001-0001-0001-00000000033c
vdsClient -s frodo createVolume 97338d5c-9b6f-1859-b827-e977ed082e53
00000001-0001-0001-0001-00000000033c
0288f410-71f1-4b7d-bdb7-e815a93e34ef 53690000000 5 1 2
96d7726c-6969-40a5-84bd-0925496b6051 cli_volume
----
The last command outputs a new UUID, but there is nothing under
/rhev/data-center/00000001-0001-0001-0001-00000000033c/97338d5c-9b6f-1859-b827-e977ed082e53/images/
directory.
If I pass createVolume an existing image/storage domain UUID, the
command still outputs a new UUID, but nothing changes under the images
directory.
Is there a step that I am missing somewhere in this process?
--
Hayley Swimelar
LINBIT | Keeping the Digital World Running
DRBD — Corosync — Pacemaker
+1-503-573-1262 x212
7 years, 11 months
- 6
- 12
Empty Dropdown Menus in Webadmin
by Phillip Bailey
Hi,
Is anyone else experiencing empty dropdown menus in webadmin? I started
having the problem after rebasing Friday afternoon. I rebased again this
morning, but am still experiencing the same problem. I'm trying to
determine whether this is something isolated to my environment, or a more
widespread issue.
I appreciate any advice any of you may be able to offer.
Thanks!
-Phillip Bailey
7 years, 11 months
- 3
- 3