[ovirt-devel] Subject: [ OST Failure Report ] [ master ] [20-09-2017 ] [ post-002_bootstrap.py ]

[Juan Hernández]

I am aware of that build failure. The tests of the *Ruby* SDK hang 
apparently random in ppc64le environments. I am trying to find out why, 
but not making much progress, to be honest.

Meanwhile I wonder why this is a failure to run OST tests, as the Ruby 
SDK isn't used at all in those tests, as far as I know.

On 09/20/2017 04:09 PM, Dafna Ron wrote:
> We just had a second failure which CQ is reporting to be from the same
> patch:
> 
> failed job:
> http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/console
> 
> if you look at the console output you can see this:
> 
> *12:27:15* ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le (122) failed building
> 
> This is the job that failed on build: http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le/122/
> 
> I re-ran the sdk build and it succeeded this time: http://jenkins.ovirt.org/job/ovirt-engine-sdk-ruby_4.1_build-artifacts-fc24-ppc64le/123/
> 
> please try to re-submit the patch? maybe we have a coincident but if it fails again it would be 3ed time...
> 
> Thanks,
> Dafna
> 
> 
> 
> On 09/20/2017 02:20 PM, Ondra Machacek wrote:
>> The http returned 503:
>>
>> 192.168.201.1 - - [20/Sep/2017:05:17:10 -0400] "GET
>> /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
>> 200 5249
>> 192.168.201.1 - - [20/Sep/2017:05:17:13 -0400] "GET
>> /ovirt-engine/api/hosts/06b9cdf3-5bcb-4be8-9e11-8ad5123749f0 HTTP/1.1"
>> 503 299
>> 192.168.201.4 - - [20/Sep/2017:05:17:11 -0400] "POST
>> /ovirt-engine/sso/oauth/token-info HTTP/1.1" 500 527
>>
>> At that time postgres was killed by oom:
>>
>> Sep 20 05:17:15 lago-basic-suite-master-engine kernel: postgres
>> invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
>>
>>
>> So from Python SDK perpepective all looks OK.
>>
>>
>> On Wed, Sep 20, 2017 at 2:11 PM, Martin Perina <mperina at redhat.com> wrote:
>>> Ondro, could you please take a look?
>>>
>>> On Wed, Sep 20, 2017 at 1:07 PM, Dafna Ron <dron at redhat.com> wrote:
>>>> Hi,
>>>>
>>>> We have a failure in test post-002_bootstrap.py
>>>>
>>>> from the logs I can see that lago has an unhandled exception from ovirt
>>>> sdk which makes sense since the patch reported on CQ is a change to the sdk.
>>>>
>>>> Link to suspected patches: https://gerrit.ovirt.org/#/c/82022/
>>>>
>>>>
>>>> Link to Job:
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734
>>>>
>>>>
>>>> Link to all logs:
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/
>>>>
>>>>
>>>> (Relevant) error snippet from the log:
>>>>
>>>> <error>
>>>>
>>>>
>>>> 2017-09-20
>>>> 09:17:15,676::testlib.py::assert_equals_within::227::ovirtlago.testlib::ERROR::
>>>> * Unhandled exception in <function _host_is_up_4 at 0x3dd25f0>
>>>> Traceback (most recent call last):
>>>>    File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219,
>>>> in assert_equals_within
>>>>      res = func()
>>>>    File
>>>> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
>>>> line 338, in _host_is_up_4
>>>>      host_status = host_service.get().status
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
>>>> 31529, in get
>>>>      return self._internal_get(headers, query, wait)
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>>> 202, in _internal_get
>>>>      return future.wait() if wait else future
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53,
>>>> in wait
>>>>      return self._code(response)
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>>> 199, in callback
>>>>      self._check_fault(response)
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>>> 121, in _check_fault
>>>>      body = self._internal_read_body(response)
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
>>>> 307, in _internal_read_body
>>>>      self._connection.check_xml_content_type(response)
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
>>>> 800, in check_xml_content_type
>>>>      response.headers
>>>>    File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line
>>>> 840, in _check_content_type
>>>>      raise Error(msg)
>>>> Error: The response content type 'text/html; charset=iso-8859-1' isn't the
>>>> expected XML
>>>> 2017-09-20
>>>> 09:17:15,683::utils.py::wrapper::480::lago.utils::DEBUG::Looking for a
>>>> workdir
>>>>
>>>>
>>>> </error>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Dafna
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
> 
>