[ovirt-devel] OST 4.1 failure: Error: ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running', ))

Thu May 11 18:03:27 UTC 2017

On Thu, May 11, 2017 at 7:34 PM, Anton Marchukov <amarchuk at redhat.com>
wrote:

> Hello All.
>
> We checked the OST and so far it looks like it is correct for master. It
> does use the latest SDK4 version built by the job triggered as part of the
> fix [1]. It is visible at console log [2]:
>
> *15:50:44* [basic_suit_el7] Updated:*15:50:44* [basic_suit_el7]   python-ovirt-engine-sdk4.x86_64 0:4.2.0-1.a0.20170511git210c375.el7.centos
>
>
This is incorrect version. The correct one is:

python-ovirt-engine-sdk4-4.2.0-1.a0.20170511gitcd0adb4.el7.centos.x86_64.rpm
<http://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_master_build-artifacts-el7-x86_64/71/artifact/exported-artifacts/python-ovirt-engine-sdk4-4.2.0-1.a0.20170511gitcd0adb4.el7.centos.x86_64.rpm>

>From this build:

http://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_master_build-artifacts-el7-x86_64/71/

>
> Also checking the fix at [1] I see that according to the stacktrace from
> [2] we fail at send():
>
> *15:56:58* [basic_suit_el7] Error while running thread*15:56:58* [basic_suit_el7] Traceback (most recent call last):*15:56:58* [basic_suit_el7]   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue*15:56:58* [basic_suit_el7]     queue.put({'return': func()})*15:56:58* [basic_suit_el7]   File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 327, in _add_host_4*15:56:58* [basic_suit_el7]     name=CLUSTER_NAME,*15:56:58* [basic_suit_el7]   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 8726, in add*15:56:58* [basic_suit_el7]     return self._internal_add(host, headers, query, wait)*15:56:58* [basic_suit_el7]   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_add*15:56:58* [basic_suit_el7]     context = self._connection.send(request)*15:56:58* [basic_suit_el7]   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 300, in send*15:56:58* [basic_suit_el7]     sys.exc_info()[2]*15:56:58* [basic_suit_el7]   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 295, in send*15:56:58* [basic_suit_el7]     return self.__send(request)*15:56:58* [basic_suit_el7]   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 413, in __send*15:56:58* [basic_suit_el7]     self._multi.add_handle(curl)*15:56:58* [basic_suit_el7] Error: ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running',))
>
>
> While at the fix [1] I see the lock is added to wait() method while
> according to the stacktrace we have we fail at send() method and as I see
> the wait() is executed by _internal_add() later. So that code added at [1]
> is not executed yet.
>

The send() method already has a lock.

>
> Do we have any other fix that we have missed? Just want to make sure we
> track the correct gerrit fix through our system.
>

This patch should be correct fix.

>
> Anton.
>
> [1] https://gerrit.ovirt.org/#/c/76713/
> [2] http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6643/
> consoleFull
>
>
>
> On Thu, May 11, 2017 at 5:31 PM, Eyal Edri <eedri at redhat.com> wrote:
>
>> We are aware of this failure.
>> Initially it was a bug in SDK4 and was (probably) fixed by Ondra,
>> But now we have another bug in repoman which takes an older SDK version
>> instead of latest.
>>
>> We're working on a fix as we speak.
>>
>> On Thu, May 11, 2017 at 6:26 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>>
>>> I got this failure in 4.1 build[1], which should not be relevant to
>>> the tested patch[2] - is this a known issue?
>>>
>>> *13:31:47*   # add_hosts: *13:31:47* Error while running thread*13:31:47* Traceback (most recent call last):*13:31:47*   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue*13:31:47*     queue.put({'return': func()})*13:31:47*   File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py", line 320, in _add_host_4*13:31:47*     name=CLUSTER_NAME,*13:31:47*   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 8726, in add*13:31:47*     return self._internal_add(host, headers, query, wait)*13:31:47*   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_add*13:31:47*     context = self._connection.send(request)*13:31:47*   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 300, in send*13:31:47*     sys.exc_info()[2]*13:31:47*   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 295, in send*13:31:47*     return self.__send(request)*13:31:47*   File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 413, in __send*13:31:47*     self._multi.add_handle(curl)*13:31:47* Error: ('Error while sending HTTP request', error('cannot add/remove handle - multi_perform() already running',))
>>>
>>>
>>> [1] http://jenkins.ovirt.org/view/oVirt%20system%20tests/job
>>> /ovirt-system-tests_manual/374/console
>>> [2] https://gerrit.ovirt.org/76645
>>>
>>> Nir
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>>
>> --
>>
>> Eyal edri
>>
>>
>> ASSOCIATE MANAGER
>>
>> RHV DevOps
>>
>> EMEA VIRTUALIZATION R&D
>>
>>
>> Red Hat EMEA <https://www.redhat.com/>
>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>> phone: +972-9-7692018 <+972%209-769-2018>
>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>
>
>
>
> --
> Anton Marchukov
> Senior Software Engineer - RHEV CI - Red Hat
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170511/df1f6a31/attachment-0001.html>