3:30 a.m.
Eyal,
This was ioprocess issue occurring after the fix was provided. I
haven't seen it since build #1389.
Thanks,
Piotr
On Thu, May 19, 2016 at 3:00 PM, Eyal Edri <eedri(a)redhat.com> wrote:
was that resolved?
any infra issue or was it problems with the tests?
On Mon, May 16, 2016 at 3:27 PM, Piotr Kliczewski
<piotr.kliczewski(a)gmail.com> wrote:
>
> and one more:
>
>
> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1389/con...
>
> On Mon, May 16, 2016 at 1:46 PM, Piotr Kliczewski
> <piotr.kliczewski(a)gmail.com> wrote:
> > One more occurrence of the issue [1]
> >
> >
> > [1]
> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1359/console
> >
> > On Sun, May 15, 2016 at 8:37 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> >> The ioprocess issue fixed in https://gerrit.ovirt.org/57473
> >>
> >> Will be merge soon and available via ovirt-release-master.
> >>
> >> Nir
> >>
> >> On Sun, May 15, 2016 at 7:45 PM, Nir Soffer <nsoffer(a)redhat.com>
wrote:
> >>> Hi all,
> >>>
> >>> I found another stuck build today:
> >>>
> >>>
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1151/con...
> >>>
> >>> 11:27:18
> >>>
------------------------------------------------------------------------------------------------------------------------------------------------
> >>> 11:27:18 TOTAL
> >>> 40513 21121
> >>> 48%
> >>> 11:27:18
> >>> ----------------------------------------------------------------------
> >>> 11:27:18 Ran 2169 tests in 145.934s
> >>> 11:27:18
> >>> 11:27:18 OK (SKIP=88)
> >>> 11:27:18 Exception AttributeError: "'NoneType' object has
no attribute
> >>> 'write'" in <bound method IOProcess.__del__ of
<ioprocess.IOProcess
> >>> object at 0x7fd7c9f2d3d0>> ignored
> >>> [...]
> >>> 11:27:18 Exception AttributeError: "'NoneType' object has
no attribute
> >>> 'write'" in <bound method IOProcess.__del__ of
<ioprocess.IOProcess
> >>> object at 0x7fd7c9f15550>> ignored
> >>> 11:27:18 Exception in thread ioprocess communication (6533) (most
> >>> likely raised during interpreter shutdown):
> >>> 11:27:18 Traceback (most recent call last):
> >>> 11:27:18 File "/usr/lib64/python2.7/threading.py", line 804,
in
> >>> __bootstrap_inner
> >>> 11:27:18 File "/usr/lib64/python2.7/threading.py", line 757,
in run
> >>> 11:27:18 File
> >>> "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line
180, in
> >>> _communicate
> >>> 11:27:18 <type 'exceptions.AttributeError'>:
'NoneType' object has no
> >>> attribute 'close'
> >>>
> >>> This seems smells like a non-daemon thread started by some code,
> >>> blocking hte test process.
> >>>
> >>> I suspect ioprocess, starting such thread, looking into it.
> >>>
> >>> Meanwhile, please:
> >>> - verify that all threads in actual code and in the tests are daemon
> >>> threads
> >>> - convert your threads to use vdsm.concurrent.thread instead of
> >>> threading.Thread (daemon by default)
> >>> - watch your builds and abort stuck builds
> >>>
> >>> David, we need a timeout in the ci, aborting the job after a project
> >>> based timeout, maybe
> >>> defined in the project yaml.
> >>>
> >>> Cheers,
> >>> Nir
> >> _______________________________________________
> >> Devel mailing list
> >> Devel(a)ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/devel
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
>
--
Eyal Edri
Associate Manager
RHEV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)