<div dir="ltr">was that resolved? <div>any infra issue or was it problems with the tests?</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, May 16, 2016 at 3:27 PM, Piotr Kliczewski <span dir="ltr"><<a href="mailto:piotr.kliczewski@gmail.com" target="_blank">piotr.kliczewski@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">and one more:<br>
<br>
<a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1389/console" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1389/console</a><br>
<div class="HOEnZb"><div class="h5"><br>
On Mon, May 16, 2016 at 1:46 PM, Piotr Kliczewski<br>
<<a href="mailto:piotr.kliczewski@gmail.com">piotr.kliczewski@gmail.com</a>> wrote:<br>
> One more occurrence of the issue [1]<br>
><br>
><br>
> [1] <a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1359/console" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1359/console</a><br>
><br>
> On Sun, May 15, 2016 at 8:37 PM, Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>> wrote:<br>
>> The ioprocess issue fixed in <a href="https://gerrit.ovirt.org/57473" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/57473</a><br>
>><br>
>> Will be merge soon and available via ovirt-release-master.<br>
>><br>
>> Nir<br>
>><br>
>> On Sun, May 15, 2016 at 7:45 PM, Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>> wrote:<br>
>>> Hi all,<br>
>>><br>
>>> I found another stuck build today:<br>
>>> <a href="http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1151/console" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1151/console</a><br>
>>><br>
>>> 11:27:18 ------------------------------------------------------------------------------------------------------------------------------------------------<br>
>>> 11:27:18 TOTAL<br>
>>> 40513 21121<br>
>>> 48%<br>
>>> 11:27:18 ----------------------------------------------------------------------<br>
>>> 11:27:18 Ran 2169 tests in 145.934s<br>
>>> 11:27:18<br>
>>> 11:27:18 OK (SKIP=88)<br>
>>> 11:27:18 Exception AttributeError: "'NoneType' object has no attribute<br>
>>> 'write'" in <bound method IOProcess.__del__ of <ioprocess.IOProcess<br>
>>> object at 0x7fd7c9f2d3d0>> ignored<br>
>>> [...]<br>
>>> 11:27:18 Exception AttributeError: "'NoneType' object has no attribute<br>
>>> 'write'" in <bound method IOProcess.__del__ of <ioprocess.IOProcess<br>
>>> object at 0x7fd7c9f15550>> ignored<br>
>>> 11:27:18 Exception in thread ioprocess communication (6533) (most<br>
>>> likely raised during interpreter shutdown):<br>
>>> 11:27:18 Traceback (most recent call last):<br>
>>> 11:27:18 File "/usr/lib64/python2.7/threading.py", line 804, in<br>
>>> __bootstrap_inner<br>
>>> 11:27:18 File "/usr/lib64/python2.7/threading.py", line 757, in run<br>
>>> 11:27:18 File<br>
>>> "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 180, in<br>
>>> _communicate<br>
>>> 11:27:18 <type 'exceptions.AttributeError'>: 'NoneType' object has no<br>
>>> attribute 'close'<br>
>>><br>
>>> This seems smells like a non-daemon thread started by some code,<br>
>>> blocking hte test process.<br>
>>><br>
>>> I suspect ioprocess, starting such thread, looking into it.<br>
>>><br>
>>> Meanwhile, please:<br>
>>> - verify that all threads in actual code and in the tests are daemon threads<br>
>>> - convert your threads to use vdsm.concurrent.thread instead of<br>
>>> threading.Thread (daemon by default)<br>
>>> - watch your builds and abort stuck builds<br>
>>><br>
>>> David, we need a timeout in the ci, aborting the job after a project<br>
>>> based timeout, maybe<br>
>>> defined in the project yaml.<br>
>>><br>
>>> Cheers,<br>
>>> Nir<br>
>> _______________________________________________<br>
>> Devel mailing list<br>
>> <a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
>> <a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>
_______________________________________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>
<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Eyal Edri<br>Associate Manager</div><div>RHEV DevOps<br>EMEA ENG Virtualization R&D<br>Red Hat Israel<br><br>phone: +972-9-7692018<br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div>
</div>